Data Lake for Enterprises

Flume source

Flume agent can have multiple sources, but it is mandatory to have at least one source for it to function. The source is managed by Source Runner which controls the threading aspect and execution models namely:

• Event-driven and
• Polling

In event-driven execution model the source listens and consumes events. In polling execution model the source keeps polling for events and then deal with it.

The event (as detailed earlier) can take a variety of content satisfying the event schema (header and payload). The source, complying with the architecture principle of extensibility, works on plugin approach. The source requires mandated name and type. According to the type, source will demand additional parameters and accordingly configurations ...