Users can run Kafka Connect in two ways: standalone mode or distributed mode.
In standalone mode, a single process runs all the connectors. It is not fault tolerant. Since it uses only a single process, it is not scalable. Generally, it is useful for users for development and testing purposes.
In distributed mode, multiple workers run Kafka Connect. In this mode, Kafka Connect is scalable and fault tolerant, so it is used in production deployment.
Let's learn more about Kafka and Kafka Connect (standalone mode). In this example, we will do the following:
- Install Kafka
- Create a topic
- Generate a few messages to verify the producer and consumer
- Kafka Connect-File-source and file-sink ...
