Design principles
Kafka is run as a cluster of one or more servers that act as message brokers (https://en.wikipedia.org/wiki/Message_broker) in the system. Kafka categorizes the stream of records under topics that are used by producers and consumers to produce records and consume them, respectively. Each record consists of a key-value pair and a timestamp.
A typical Kafka cluster is shown in the following diagram along with the structure of a Kafka topic and its partitions. A topic is a core abstraction for a stream of records in Kafka. Producers publish the records to a topic that can have zero, one, or more consumers subscribed to it. Each topic consists of one or more partitions that are ordered, immutable sequence of records that is ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access