Data Lake for Enterprises

Data acquisition of stream data - technology mapping

The following figure brings in technology aspect to the conceptual architecture that we will be the following throughout this book.

We have chosen Apache Flume as the real time data transfer technology and this does come in the data acquisition layer of our Data Lake implementation.

Figure 02: Technology mapping for Acquisition Layer

Inline with our use case of SCV, the real time data from various business applications will flow into the Flume and then transferred to the Hadoop file system for storage and later analysis. The real time data from business application that we are going to handle ...