Understanding the responsibilities of Kafka consumers

On the same lines of the previous chapter on Kafka producers, we will start by understanding the responsibilities of Kafka consumers apart from consuming messages from Kafka queues.

Let's look at them one by one:

  • Subscribing to a topic: Consumer operations start with subscribing to a topic. If consumer is part of a consumer group, it will be assigned a subset of partitions from that topic. Consumer process would eventually read data from those assigned partitions. You can think of topic subscription as a registration process to read data from topic partitions.
  • Consumer offset position: Kafka, unlike any other queues, does not maintain message offsets. Every consumer is responsible for ...

Get Building Data Streaming Applications with Apache Kafka now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.