Learning Apache Apex
by Ananth Gundabattula, Thomas Weise, Munagala V. Ramanath, David Yan, Kenneth Knowles
What to checkpoint
As described, checkpointing is costly and when writing custom operators, it is important to consider what data needs to be part of the saved state.
Some operators are stateless; their computation does not depend on the previous state. Application developers can skip the checkpointing for such operations using the STATELESS attribute, or if it is a custom operator, by annotating the operator with @Stateless. Examples include stateless transformations such as filtering or parsing or a connector that writes to JDBC (refer to Chapter 3, The Apex Library).
Stateful operators, such as the counter discussed previously, depend for their computation on state from previous streaming windows and need to be checkpointed. If there are ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access