Apache Flume: Distributed Log Collection for Hadoop

Summary

In this chapter we covered the following various interceptors shipped with Flume:

• Timestamp: This is used to add a timestamp header, possibly overwriting an existing one.
• Host: This is used to add the Flume agent host name or IP as a header in the event.
• Static: This is used to add static String headers.
• Regular expression filtering: This is used to include or exclude events based on a matched regular expression.
• Regular expression extractor: This is used to create headers from matched regular expression headers. It is also useful for routing with channel selectors.
• Custom: This is used to create any custom transformations you need that you can't find elsewhere.

We also covered tiering data flows using the Avro Source and Sink.

Next we introduced ...