Data Lake for Enterprises

Serving layer

The core task of the serving layer is to expose the views created by both the batch and speed layer for querying by other systems or users.

Apart from this, there is a good amount of orchestration work done by this layer. The speed layer needs to keep looking at the batch layer to see if the necessary batch operation is done. If so, the batch layer store needs to be updated soon after the batch operation is conducted from the batch view. Soon after this, the existing speed layer view will be discarded and this is also a bookkeeping activity which keeps the speed layer store size in control.

The following figure shows the two layers namely batch and serving in action:

Figure 09: Lambda Architecture - serving layer

When a query ...