Data Lake for Enterprises

Advanced Flume architecture

The Flume architecture in an enterprise setting will have one more important component namely Flume master, whose main responsibility is to serve as a centralized authority for all the configurations of all nodes in the overall architecture. Every machine participating in the architecture is termed as node. Each node depends on the master to retrieve the configurations which dictates as to how the Flume should perform its actions.

The entire Flume topology can be configured or reconfigured dynamically by sending relevant commands using Thrift API to the Flume master.

The following figure (Figure 12) shows an advanced Flume architecture arrangement, which shows the all-important Flume master controlling the configuration ...