Learning Apache Apex
by Ananth Gundabattula, Thomas Weise, Munagala V. Ramanath, David Yan, Kenneth Knowles
Triggering in Beam
Triggering in Beam governs when output is emitted by aggregations in Beam. It is not necessary to wait for your data to be complete, nor is it required to emit output when data is complete. Completeness is merely one input to the decision about whether an aggregation should emit a result.
More specifically, triggering aims at letting you control the trade-off, for your pipeline, between completeness of your results, latency of output, and cost due to excessive recomputation. Let's consider these dimensions for three common use cases: a monthly billing pipeline, a running total of a monthly bill prior to sending it, and an abuse detection pipeline. In the following figure, the blue column indicates the importance of complete ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access