Data Lake for Enterprises

Kafka consumer

Consumers consume messages from the topic from within the Kafka cluster. It consumes messages in a sequential manner from within a partition of a topic. The consumer is required to make sure that it tracks the messages being taken/read from the topic and when asked for a message it is required to supply a sequential ID and the messages from that ID onwards are read in a sequential manner thereafter. Kafka uses a pull approach from the consumer to consume a message and this is one of the main reason for the high throughput of message consumption in Kafka. Also, the throughput of message consumption is left to individual consumers, and because of this Kafka is ready to integrate with a variety of consumers having different capabilities. ...