Data Lake for Enterprises

Context for Data Lake - Data Storage and lambda Batch layer

In our Data Lake implementation, we have a dedicated layer where the data permanently resides and this is the Data Storage Layer. The data gathered from various sources is persisted in various stores capable of handling different types and forms of data. In this chapter, we are storing non-indexed raw data in our Data Lake.

We have chosen Apache Hadoop as our technology for this data storage capability. I am sure there was not much debate when we chose this technology in this layer, obviously because of the fantastic features this technology. Also, the level of maturity and support this technology possesses is quite astonishing over the short span of its existence.

The following ...