Data Lake for Enterprises

Batch and stream data

The following table (Table 01) summarizes the main difference which exist between batch and stream data. Our use case does have both these types of data. For handling batch data we selected and used Sqoop as the technology and for stream data handling and transfer to Hadoop we have selected Flume as the choice.

Batch Data	Stream Data
Mostly the volume of data is quite high and deal with a chunk of data in defined period of time. Batch consisting of thousands of records.	The volume of data is not that great. But, deal with high velocity data and it comes continuously but does not have defined period in which they come. Stream data consist of single or few record. Few records are sometimes also called as micro-batch ...