Reliability of data processing

One of the USPs of Storm is guaranteed message processing that makes it a very lucrative solution. Having said that, we as programmers have to make certain modeling to use or not use to the reliability provided for by Storm.

First of all, it's very important to understand what happens when a tuple is emitted into the topology and how its corresponding DAG is constructed. The following diagram captures a typical case in this context:

Reliability of data processing

Here, the function of the topology is very clear: every emitted tuple has to be filtered, calculated, and written to the HDFS and database. Now, let's take an implication of DAG with respect ...

Get Real-Time Big Data Analytics now with O’Reilly online learning.

O’Reilly members experience live online training, plus books, videos, and digital content from 200+ publishers.