Data Lake for Enterprises

Flume configuration

Flume can be fully configured using the flume configuration file. A single image speaks more than thousand words, so we will like to explain Flume configuration using the following figure. An exhaustive flume configuration is out of scope of this book, but will explain some core aspects of how the flume can be configured and this can be base for understanding a full-fledged configuration.

Figure 17: Flume Configuration Tree (sample)

The next code block shows the preceding configuration tree figure:

# Active Flume ComponentsflumeAgent1.sources=source1flumeAgent1.channels=channel1flumeAgent1.sinks=sink1# Define and Configure ...