Building Data Streaming Applications with Apache Kafka

Message producers

In Kafka, the producer is responsible for sending data to the partition of the topic for which it is producing data.

The producer generally does not write data to partitions, it creates write requests for messages and sends them to the leader broker. Partitioner calculates the hash value of the message, which helps the producer to choose which partition should be selected.

The hash value is generally calculated by the message key that we provide when writing the message to a Kafka topic. The message with a null key will be distributed in a round-robin fashion across partitions to ensure even distribution of messages. In Kafka, each partition has a leader and each read write request goes through the leader only. So a request ...