© Deepak Vohra 2016

Deepak Vohra, Practical Hadoop Ecosystem, 10.1007/978-1-4842-2199-0_9

9. Apache Kafka

Deepak Vohra

(1)Apt 105, White Rock, British Columbia, Canada

Apache Kafka is publish-subscribe, high-throughput, distributed messaging system. Kafka is fast with a single broker handling hundreds of MB (terabytes)/sec of reads and writes from several clients. Messages are replicated across the cluster and persisted to disk. Kafka can be used for stream processing, web site activity tracking, metrics collection and monitoring, and log aggregation.

The main components of Kafka architecture are Producer, Broker, Topic, and Consumer. Kafka keeps feeds of messages in topics. Producers send (or write) messages to topics and consumers, who subscribe ...

Get Practical Hadoop Ecosystem: A Definitive Guide to Hadoop-Related Frameworks and Tools now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.