Data Lake for Enterprises

Flume in purview of SCV use case

As discussed with the overall data lake architecture in previous chapters and the Single Customer View use case, it is evident that capabilities of Flume can be leveraged in multiple ways. Flume can be used for data acquisition as well as can play a role in data ingestion as well. For the context of this chapter we will focus on data acquisition capabilities of flume from various sources in context of SCV use case.

While we may discuss the messaging layer in a greater detail in later chapters, in order to complete the data acquisition mechanisms for flume, we will also cover in the examples some basic parts of messaging engine, Apache Kafka.