Data Lake for Enterprises

Flume reliability level

Reliability is one of the core architecture principle based on which Flume architecture was designed. To achieve this level of reliability, Flume provides its user with configurable reliability levels. They are classified as follows:

• End-to-end: When this reliability level is set, the event sent to Flume will surely make it to the other end as long as the originating agent (the agent which accepted the venet) is alive. To achieve this reliability level, the agent when receiving an event stores/writes in the disk in WAL (Write Ahead Log). When the event reaches the defined endpoint and acknowledgement is sent all the way to the originating agent and then the written event is erased. This level can withstand failure ...