Setting up Kafka
At the time of this writing, the stable version of Kafka is 0.8.1. The prerequisites for running Kafka is a ZooKeeper ensemble and Java Version 1.6 or above. Kafka comes with a convenience script that can start a single-node ZooKeeper, but it is not recommended to use it in a production environment. We will be using the ZooKeeper cluster we deployed in the Setting up a ZooKeeper cluster section of Chapter 2, Setting Up a Storm Cluster.
We will see both how to set up a single-node Kafka cluster first and how to add two more nodes to it to run a full-fledged three-node Kafka cluster with replication enabled.
Setting up a single-node Kafka cluster
The following are the steps to set up a single-node Kafka cluster:
- Download the Kafka 0.8.1.1 ...