Data Lake for Enterprises

Consumer groups

Consumer groups allow multiple hosts to form a group (using the same group) to access a particular topic. Consumer groups guarantee that a message is only read by one consumer in the group. As shown in the following figure, partitions are assigned to each consumer in a group and these are entrusted to read the messages in a topic. A consumer may get one or more partitions from which to read the messages, but it's care is taken that one partition is only assigned to a single consumer in a group.

This figure shows the working of a consumer group in action in a Kafka deployment.

Figure 09: Multiple consumers logically grouped ...