7

Apache Kafka for Real-Time Events and Data Ingestion

Real-time data and event streaming are crucial components of modern data architectures. By leveraging systems such as Apache Kafka, organizations can ingest, process, and analyze real-time data to drive timely business decisions and actions.

In this chapter, we will cover Kafka’s fundamental concepts and architecture that enable it to be a performant, resilient, and scalable messaging system. You will learn how Kafka’s publish-subscribe messaging model works with topics, partitions, and brokers. We will demonstrate Kafka setup and configuration, and you will get hands-on experience with producing and consuming messages for topics.

Additionally, you will understand Kafka’s distributed and ...

Get Big Data on Kubernetes now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.