Building Data Streaming Applications with Apache Kafka

Capacity planning goals

This is the most important thing while performing capacity planning of your Kafka cluster. You should be very clear with your capacity planning goals. You should understand that without having clear goals in mind, it is very difficult to perform appropriate capacity planning.

Generally, capacity planning goals are driven by latency and throughput. Some of the additional goals could be fault tolerance and high availability.

We suggest you derive quantifiable goals so that you can always come up with logical mathematical conclusions to capacity planning numbers. Moreover, your goals should also consider future data growth or increase in the number of requests.