Data Lake for Enterprises

Working of Kafka

Kafka’s architecture in whole is quite simplistic in nature and has some very important components, which are the crux of the whole workings of it.

The following figure shows the workings of Kafka with all components labelled. The main components which constitute Kafka architecture are as follows:

• Message: Flows from producer all the way to consumer through topics existing in a broker.
• Producer: Producer, as the name suggests produces these stream messages and pumps into the topic.
• Topic: The category into which the producer pumps the message. The producer produces messages of a particular category which fall into this topic. Ideally, for each category of messages a new topic is created in Kafka.
• Broker: Kafka cluster comprising ...