Implementing a rolling window topology
In many temporal applications, it is important to be able answer the question of "What happened in the last X amount of time?" This is commonly referred to as a sliding window:
However, while working with Storm as the Speed layer of the Lambda architecture, it is required that we implement a rolling time window whereby we can segment time in a fixed manner. These fixed-time boundaries allow us to easily merge the Batch and Speed layers and provide a complete and seamless answer.
The following diagram illustrates a rolling window:
In Chapter 6, Integrating Storm and Hadoop, we will explore combining the results ...