Data Lake for Enterprises

Flume event - Stream Data

Event is the unit of data which is send across the Flume pipeline. The structure of the event is quite simple and had two parts to it namely:

• Event header: A Key/Value pair in the form Map<String, String>. These headers are meant to add more data about the event. For example, these headers can hold severity and priority aspects of this event, and so on. These headers can also contain UUID or event ID which distinguishes one event from the other.
• Event payload: An array of bytes (byte array) in the form byte[]. 32 KB is the default body size, which is usually truncated after that figure but this is a configurable value in Flume.

This figure shows the internal structure of the Flume event, which hops from one agent ...