Integrating a non-transactional system
To extend on our previous example, we could develop a framework for the configuration that would allow the user to specify the dimensions along which they would like to aggregate events. Then, we could use that configuration in our topology to maintain a set of in-memory data sets to accumulate the aggregations, but any in-memory store is susceptible to faults. To address fault-tolerance, we could then make those aggregations persist in a database.
We would need to anticipate and support all the different types of aggregations the user would like to perform (for example, sum, average, geospatial, and so on). This seems like a substantial endeavor.
Fortunately, there are options for real-time analytics engines. ...
Get Storm Blueprints: Patterns for Distributed Real-time Computation now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
