The ETL is the processing of extracting data from the source system, doing some transformation, and loading data to the target system. In the last couple of years, development around Kafka has been moving fast and there has been effort to make Kafka a unified model for building your ETL pipeline. Kafka Connect and Kafka Streams are two of them, which can help you build your ETL pipeline. Kafka Connects are of two types; one is the Source connector and other one is the Sink connector. The source connector is responsible for bringing data to Kafka and the Sink connector is used to move data out of Kafka. Let's look at the following diagram for more information:
This diagram indicates the three major steps involved in ...
