Data Lake for Enterprises

Speed layer

The speed layer in a Lambda Architecture provides near real-time processing of events. Since the expectation is to process the events in near-real-time there is a limited amount of processing that can be done on a limited size of information. This may also include machine learning or complex event processing algorithms that can be run for near-real-time scenarios.

The term near-real-time processing is a relative term and may mean different things for different people and different scenarios. For instance, for a customer reservation, this may mean of the order of 2-3 seconds, however for a use case such as recommendation engine, it may mean a few minutes.

In terms of Lambda Architecture, this layer should receive the same event/message ...