Data Lake for Enterprises

Sink Processor

Sink Processor dictates how the Sink Group will function and achieve the load balancing or failover scenarios required by the reliability guarantee agreed for your Flume setup. Sink Processor is also a top level component in the Flume configuration. Broadly Sink Processor is classified into two:

1. Built-in Sink Processor: These are processors present by default with Apache Flume.
   • Default Sink Processor:
     • Accepts only one sink.
     • Doesn't have to be explicitly put as a single sink has this processor by default.
   • Failover Sink Processor:
     • Keeps a prioritized list of sinks
     • Uses that priority to select the sink and makes sure that there is always a sink to process an event.
     • If an event fails while sending to a sink, the next event ...