Apache Kafka

How to reset the consumer offset in Apache Kafka topic

How to use kafka-consumer-groups.sh to reset topic offsets

Bartosz Mikulski 20 Dec 2020 – 1 min read

In Kafka, we can easily reset the consumer group offset to any value we want. It allows us to reprocess some messages or skip the messages we don’t want to process. All we need is the kafka-consumer-groups.sh.

For example, if I want to reset the offset of the topic my_topic accessed by the consumer group called the_consumers to the earliest available offset, I have to run the following command:

kafka-consumer-groups.sh --bootstrap-server <kafka_broker_host:9091> --group
    the_consumers --reset-offsets --to-earliest --topic my_topic --execute

Note that I must add the --execute parameter at the end of the command. Otherwise, it will print the result of the operation, but it will not perform it!

There are a few more options regarding resetting the offsets. In my opinion, the most useful ones are:

to the latest, which makes the consumers skip all of the existing messages and start with the last one added to the topic:

kafka-consumer-groups.sh --bootstrap-server <kafka_broker_host:9091> --group
    the_consumers --reset-offsets --to-latest --topic my_topic --execute

to datetime, which sets the offset to the offset of the first message added to the topic after the datetime:

kafka-consumer-groups.sh --bootstrap-server <kafka_broker_host:9091> --group
    the_consumers --reset-offsets --to-datetime 2020-12-20T00:00:00.000 --topic my_topic --execute

Note that it uses the date in the ISO8601 format.

Older post

How to reset the consumer offset in Apache Kafka topic

How to purge a Kafka topic

How to add dependencies as jar files or Python scripts to PySpark

How to reset the consumer offset in Apache Kafka topic

How to purge a Kafka topic

How to add dependencies as jar files or Python scripts to PySpark

Related Posts

How to purge a Kafka topic

How does Kafka Connect work?

What is Kafka log compaction, and how does it work?