April 2015
Beginner to intermediate
328 pages
11h 1m
English
This chapter covers
In the last chapter you learned the core concepts of micro-batch processing. By processing tuples in a series of small batches, you can achieve exactly-once processing semantics. By maintaining a strong ordering on the processing of batches and storing the batch ID information with your state, you can know whether or not the batch has been processed before. This allows you to avoid ever applying updates multiple times, thereby achieving exactly-once semantics.
You saw how with some minor extensions pipe diagrams could be used to represent ...