Machine Learning, 2nd Edition

APPENDIX AKafka Quick Start

For a full explanation of how Kafka works and the full walk-through of producers and consumers, please look at Chapter 12.

You can download the open source version of Kafka from this link:

https://www.apache.org/dyn/closer.cgi?path=/kafka/2.2.0/kafka_2.12-2.2.0.tgz

Installing Kafka

The downloaded distribution comes with both Zookeeper and Kafka and all the command-line tools explained in this appendix. Before you can use it, create a directory to work from and untar the distribution file.

$ tar -zxf kafka_2.12-2.2.0.tgz
$ cd kafka_2.12-2.2.0

Starting Zookeeper

For development, a stand-alone cluster will work fine, so there are no changes required to any of the configuration files. The first thing to start is Zookeeper. In a terminal window, run the following as the root user (if you don’t use the root user, you may get file permission issues):

$ bin/zookeeper-server-start.sh config/zookeeper.properties
[2019-06-18 07:43:12,238] INFO Reading configuration from:
config/zookeeper.properties (org.apache.zookeeper.server.quorum.
QuorumPeerConfig)
[2019-06-18 07:43:12,241] INFO autopurge.snapRetainCount set to 3
(org.apache.zookeeper.server.DatadirCleanupManager)
[2019-06-18 07:43:12,241] INFO autopurge.purgeInterval set to 0
(org.apache.zookeeper.server.DatadirCleanupManager)
[2019-06-18 07:43:12,243] INFO Purge task is not scheduled.
(org.apache.zookeeper.server.DatadirCleanupManager)
[2019-06-18 07:43:12,243] WARN Either no config or no quorum ...