Building Data Streaming Applications with Apache Kafka

Consumer metrics

Consumers are responsible for consuming data from topic and doing some processing on it, if needed. Sometimes, your consumer may be slow, or it may behave unacceptably. The following are some important metrics that will help you identify some parameters that indicate optimization on the consumer side:

• records-lag-max: The calculated difference between the producer's current offset and the consumer's current offset is known as record lag. If the difference is very big, it's fairly indicative of the consumer processing data much slower than the producer. It sends alerts for suitable action to fix up this issue, either by adding more consumer instance or by increasing partitions and increasing consumers simultaneously.
• bytes-consumed-rate ...