Data Lake for Enterprises

Flume agent configurations

Some of these aspects have been unintentionally discussed in details in the Flume architecture section, however we thought that separate section for these agent configuration is required. Since we don't want to repeat ourselves, we will be referring some aspects back to that section.

The following are main configurations using which agents can be arranged:

• Multi-hop (multi-agent flow): Similar to the one given in the preceding section Distributed pipeline - Flume architecture. One agent feeds the event to another (called as a hop) and this carries on according to use case demand. Figure 15 shows multi-hop flume agent configuration:

Figure 15: Multi-hop flume agent configuration

• Consolidation (converging flow) ...