O'Reilly logo

Storm Blueprints: Patterns for Distributed Real-time Computation by Brian O'Neill, P. Taylor Goetz

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Integrating a non-transactional system

To extend on our previous example, we could develop a framework for the configuration that would allow the user to specify the dimensions along which they would like to aggregate events. Then, we could use that configuration in our topology to maintain a set of in-memory data sets to accumulate the aggregations, but any in-memory store is susceptible to faults. To address fault-tolerance, we could then make those aggregations persist in a database.

We would need to anticipate and support all the different types of aggregations the user would like to perform (for example, sum, average, geospatial, and so on). This seems like a substantial endeavor.

Fortunately, there are options for real-time analytics engines. ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required