Fast Data Processing Systems with SMACK Stack

Kafka and Akka

The connector is available at Maven Central for Scala 2.11 in the coordinates:

libraryDependencies += "com.typesafe.akka" %% "akka-stream-kafka" % "0.11-M4"

If you remember, a producer publishes messages to Kafka topics. The message itself contains information about what topic and partition to publish, so one can publish to different topics with the same producer. The underlying implementation uses the KafkaProducer.

When creating a consumer stream we need to pass ProducerSettings defining:

• Kafka cluster bootstrap servers
• Serializers for the keys and values
• Tuning parameters

Here we have a ProducerSettings example:

import akka.kafka._ import akka.kafka.scaladsl._ import org.apache.kafka.common.serialization.StringSerializer import org.apache.kafka.common.serialization.ByteArraySerializer ...