Before running our application, we need to start a Kafka cluster. For the sake of simplicity, we are just going to start a single broker on your workstation. If you wish to set up a multinode cluster, please refer to the Kafka documentation:
- Download Kafka 0.10.2.2 from this URL: https://www.apache.org/dyn/closer.cgi?path=/kafka/1.1.1/kafka_2.11-1.1.1.tgz. We use the version 1.1.1 because we have tested it with thespark-sql-kafka-0.10 library at the time of writing. You could use a later version of Kafka, but it is not guaranteed that an old Kafka client can always communicate with a more recent broker.
- Open a console and then decompress the package in your favorite directory:
tar xfvz kafka*.tgz.
- Go to the installation ...