Data Lake for Enterprises

Flume transaction management

Throughout the previous sections we have indeed transaction aspects at various stages. The following figure summarizes these discussions in a more pictorial fashion:

Figure 18: Transaction management in Flume (Source Tx and Sink Tx)

This figure shows that incoming data from a client or previous sink starts the present agent transaction and this is termed as Source Tx in the figure. The Source Tx ends soon after the event is persisted in the channel and acknowledgement received.

In purview of an agent a second transaction kicks in termed as Sink Tx which start with the data being polled by the sink and when the ...