Chapter 6. Kafka Internals
It is not strictly necessary to understand Kafka’s internals in order to run Kafka in production or write applications that use it. However, knowing how Kafka works does provide context when troubleshooting or trying to understand why Kafka behaves the way it does. Since covering every single implementation detail and design decision is beyond the scope of this book, in this chapter we focus on a few topics that are especially relevant to Kafka practitioners:
How Kafka replication works
How Kafka handles requests from producers and consumers
How Kafka handles storage such as file format and indexes
Understanding these topics in-depth will be especially useful when tuning Kafka—understanding the mechanisms that the tuning knobs control goes a long way toward using them with precise intent rather than fiddling with them randomly.
Kafka uses Apache Zookeeper to maintain the list of brokers that are currently members of ...