Software Architecture Patterns, Antipatterns, and Pitfalls

Chapter 14. Multi-Broker Pattern

A Note for Early Release Readers

With Early Release ebooks, you get books in their earliest form—the author’s raw and unedited content as they write—so you can take advantage of these technologies long before the official release of these titles.

This will be the 23rd chapter of the final book. Please note that the GitHub repo will be made active later on.

If you’d like to be actively involved in reviewing and commenting on this draft, please reach out to the editor at [e-mail].

Standard messaging, using queues or topics, works well for message throughput less than around 4,000 persisted messages per second. After that, architects must either use nonpersistent messages or streaming brokers (such as Kafka) to achieve higher throughput rates. Fortunately, there’s another option to increase message throughput: the Multi-Broker pattern.

Table 14-1 summarizes the metadata for the Multi-Broker pattern.

Table 14-1. Metadata for the Multi-Broker pattern

CATEGORY	METADATA
CONTEXT:	Event-Driven Architecture, Message-Based Architectures, Microservices
IMPROVED:	Elasticity, Scalability, Responsiveness, Throughput
DEGRADED:	Cost, Message Order, Simplicity

How Can I Improve Message Throughput and Capacity?

Most message brokers can handle a message throughput rate of 4,000 messages per second when using persisted messages. However, when the throughput rate exceeds this capacity, messages can back up, resulting in a loss of responsiveness, ...