Apache Flink supports various connectors that allow data read/writes across various technologies. Let's learn more about this.

Kafka connector

Kafka is a publish-subscribe, distributed, message queuing system that allows users to publish messages to a certain topic; this is then distributed to the subscribers of the topic. Flink provides options to define a Kafka consumer as a data source in Flink Streaming. In order to use the Flink Kafka connector, we need to use a specific JAR file.

The following diagram shows how the Flink Kafka connector works:

Kafka connector

We need to use the following Maven dependency to use the connector. I have been using Kafka ...

Get Learning Apache Flink now with O’Reilly online learning.

O’Reilly members experience live online training, plus books, videos, and digital content from 200+ publishers.