The need for fast data applications is growing rapidly, driven by the IoT, the surge in machine-to-machine (M2M) data, global mobile device proliferation, and the monetization of SaaS platforms. So how do you combine real-time, streaming analytics with real-time decisions in an architecture that’s reliable, scalable, and simple?
In this O’Reilly report, Ryan Betts and John Hugg from VoltDB examine ways to develop apps for fast data, using pre-defined patterns. These patterns are general enough to suit both the do-it-yourself, hybrid batch/streaming approach, as well as the simpler, proven in-memory approach available with certain fast database offerings.
Their goal is to create a collection of fast data app development recipes. We welcome your contributions, which will be tested and included in future editions of this report.
Table of contents
- Fast Data Application Value
- Fast Data and the Enterprise
- 1. What Is Fast Data?
- 2. Disambiguating ACID and CAP
- 3. Recipe: Integrate Streaming Aggregations and Transactions
- 4. Recipe: Design Data Pipelines
- 5. Recipe: Pick Failure-Recovery Strategies
6. Recipe: Combine At-Least-Once Delivery with Idempotent Processing to Achieve Exactly-Once Semantics
- Idea in Brief
- Pattern: Use Upserts Over Inserts
- Pattern: Tag Data with Unique Identifiers
- Pattern: Use Kafka Offsets as Unique Identifiers
- Example: Call Center Processing
- When to Avoid This Pattern
- Related Concepts and Techniques
- Title: Fast Data: Smart and at Scale
- Release date: October 2015
- Publisher(s): O'Reilly Media, Inc.
- ISBN: 9781491940372
You might also like
Streaming data is a big deal in big data these days. As more and more businesses …
Data Science from Scratch, 2nd Edition
To really learn data science, you should not only master the tools—data science libraries, frameworks, modules, …
Foundations for Architecting Data Solutions
While many companies ponder implementation details such as distributed processing engines and algorithms for data analysis, …
Kafka: The Definitive Guide, 2nd Edition
Every enterprise application creates data, whether it consists of log messages, metrics, user activity, outgoing messages, …