Kafka Connect

Chapter 5. Mirroring clusters with MirrorMaker

A Note for Early Release Readers

With Early Release ebooks, you get books in their earliest form—the authors’ raw and unedited content as they write—so you can take advantage of these technologies long before the official release of these titles.

This will be the sixth chapter of the final book.

If you have comments about how we might improve the content and/or examples in this book, or if you notice missing material within this chapter, please reach out to the authors at [e-mail].

The action of copying data between two Kafka clusters is called mirroring. This term is used to distinguish this process from the term “replication” that usually refers to the way data in Kafka is shared across brokers within the cluster. However both terms are often used by the community when talking about copying data between clusters.

The idea of mirroring data between clusters is pretty much as old as Kafka itself. At Kafka’s inception, mirroring was a feature of the broker, before it was then separated into its own tool in early 2012. The tool was a standalone application named MirrorMaker but due to its initial design, it had a number of limitations and was hard to operate. So in 2019, a new mirroring tool based on Kafka Connect was introduced via KIP-382 called MirrorMaker2. The initial MirrorMaker tool is now deprecated since Kafka 3.0 and will be removed in Kafka 4.0 (via KIP-720), the new tool is now often simply called MirrorMaker ...