Data Lake for Enterprises

Speed layer

Speed layer is also known as the Real-time Layer and (as the name suggests) caters to real-time analysis requirements. The batch layer operates in specified intervals and between the completion of one batch execution and start of another; the business users still wants analysis to be conducted on the data. The responsibility of merging the batch view along with real-time data is the role of the Speed layer. Now, the batch processing window can be reduced, but since batch deals with a good amount of data, processing by the batch layer usually takes time and the business cannot wait for this lag in processing of the batch layer. For achieving near real-time data for analysis, data is incremented to the speed layer in a low latency ...