Data Lake for Enterprises

Flume for data acquisition

Flume plays a very important part in the data acquisition capability in our Data Lake implementation. Below figure pictorially shows how Flume's stream processing works.

Figure 07: Flume stream processing

Shown here is a very-high-level component architecture of the Flume process, which consists of Source, Interceptors, Channels, and Sinks. Each one of these components has a very specific responsibility when handling events/messages.

While we will be dwelling on this. In much more detail in later chapters, please check out a summary of these components now.