Writing producer and consumer application

Kafka provides a rich set of APIs to write your own producer or consumer application. Kafka does has a few commonly used producer and consumers in the form of connectors, which you can run by changing the configuration. In this section, we will see how to write producer and consumer application using the Kafka API. Let's see how we can write our own producer and consumer application. The following are a few prerequisites before we jump into coding:

  • IDE: We recommend that you use a Java and Scala-supported IDE, such as IDEA, NetBeans, or Eclipse. We have used JetBrains IDEA. You can get more details from the following link: https://www.jetbrains.com/idea/
  • Build Tool: We have used Maven as the build ...

Get Mastering Hadoop 3 now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.