Building Data Streaming Applications with Apache Kafka

Out-of-order events

This is one of the key problems with any unbound data stream. Sometimes an event arrives so late that events that should have been processed after that out of order event are processed first. Events from varied remote discrete sources may be produced at the same time and, due to network latency or some other problem, some of them are delayed. The challenge with out-of-order events is that as they come very late, processing them involves data lookups on relevant datasets.

Moreover, it is very difficult to determine the conditions that help you decide if an event is an out-of-order event. In other words, it is difficult to determine if all events in each window have been received or not. Moreover, processing these out-of-order ...