Data Lake for Enterprises

Data acquisition of batch data - technology mapping

To cover our use case and to build data lakes, we use two different technologies in this layer, namely Apache Sqoop and Apache Flume. This chapter dives deep into Sqoop and Chapter 7, Messaging Layer with Apache Kafka dives deep into Flume.

The following figure brings in the technology aspect to the conceptual architecture that we will be following throughout this book. We will keep explaining each technology and its relevance in the overall architecture before we bring all the technologies together in the final part of this book (Part 3).

Figure 02: Technology mapping for acquisition layer ...