Data Lake for Enterprises

Example 3: Interceptors

There are multiple interceptors supported by Flume out of the box, and are very useful for specific scenarios. As indicated previously, the interceptors act on source events, to intercept them and add a few more details to the event as needed.

In order to better understand the usage of Interceptors, we will take the example of spooled events being streamed into kafka. We will intercept  these spooled events and add 2 more attributes which can be useful for our Data Lake:

1. Timestamp (timestamp): Adding of timestamp to every event will help maintain the time profile of event, such that an event’s end to end timing can be tracked.
2. UUID (eventId): Adding UUID to an event will help uniquely identify each event. Since an ...