May 2017
Beginner to intermediate
596 pages
15h 2m
English
Content preview from Data Lake for Enterprises
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
Start your free trial
Flume agent
Flume agent is the smallest possible deployment comprising of Source, Channel and Sink as its main components. The following figure shows a typical Flume agent deployment:
Figure 14: Flume Agent components
Flume agent is a Java daemon which received event from a source and then passes onto a channel, where it is usually written to the disk (according to reliability level set) and then moves the event to the sink. When the sink receives the event it sends acknowledgement back to channel and channel erases the event from its store. The agent has a very small memory footprint (-Xmx20m) and can be controlled declaratively using configurations. ...