Kafka producer internals

In this section, we will walk through different Kafka producer components, and at a higher level, cover how messages get transferred from a Kafka producer application to Kafka queues. While writing producer applications, you generally use Producer APIs, which expose methods at a very abstract level. Before sending any data, a lot of steps are performed by these APIs. So it is very important to understand these internal steps in order to gain complete knowledge about Kafka producers. We will cover these in this section. First, we need to understand the responsibilities of Kafka producers apart from publishing messages. Let's look at them one by one:

  • Bootstrapping Kafka broker URLs: The Producer connects to at least ...

Get Building Data Streaming Applications with Apache Kafka now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.