Setting up multi-broker cluster

While we may be setting up the Kafka server on a single node deployment in our example code, we can always have multiple broker deployment for single node level resiliency. In that case, if one broker fails, the other broker is still available to serve the messages.

Setting up multiple brokers on a single node is very straightforward and involves changing the following settings in ${KAFKA_HOME}/config/server.properties.

Configuration Parameter Description
broker.id This should always be unique in a Kafka cluster for each of the broker instances
port The port must be different if multiple brokers are to be set up on the same node
logs.dir The log location for each of the brokers must be defined on ...

Get Data Lake for Enterprises now with O’Reilly online learning.

O’Reilly members experience live online training, plus books, videos, and digital content from 200+ publishers.