Learning Apache Apex

Allowed lateness

When processing an unbounded streaming source, the number of possible windows is also unbounded (with the exception of Global Window). Theoretically, there is no limit on how late an incoming event can be. In practice, there is often a limit on how late an event can be for any meaningful processing to be done and there is always a limit on how much resource the system has for storing the state of the windows.

The WindowedOperator has a property called Allowed Lateness, which the user can specify to indicate the lateness horizon from the watermark. Events with timestamps that lie beyond the lateness horizon are dropped, and any windows that lie completely beyond the horizon are purged from the state of the operator. Thus, ...