Why have stream-oriented data systems become so popular, when batch-oriented systems have served big data needs for many years? In this report, author Dean Wampler examines the rise of streaming systems for handling time-sensitive problems—such as detecting fraudulent financial activity as it happens. You’ll explore the characteristics of fast data architectures, along with several open source tools for implementing them.
Batch-mode processing isn’t going away, but exclusive use of these systems is now a competitive disadvantage. You’ll learn that, while fast data architectures are much harder to build, they represent the state of the art for dealing with mountains of data that require immediate attention.
- Learn step-by-step how a basic fast data architecture works
- Understand why event logs are the core abstraction for streaming architectures, while message queues are the core integration tool
- Use methods for analyzing infinite data sets, where you don’t have all the data and never will
- Take a tour of open source streaming engines, and discover which ones work best for different use cases
- Get recommendations for making real-world streaming systems responsive, resilient, elastic, and message driven
- Explore an example streaming application for the IoT: telemetry ingestion and anomaly detection for home automation systems
Table of contents
- 1. Introduction
- 2. The Emergence of Streaming
- 3. Event Logs and Message Queues
- 4. How Do You Analyze Infinite Data Sets?
- 5. Real-World Systems
- 6. Example Application
- 7. Where to Go from Here
- Title: Fast Data Architectures for Streaming Applications
- Release date: October 2016
- Publisher(s): O'Reilly Media, Inc.
- ISBN: 9781491970775
You might also like
Designing Data-Intensive Applications
Data is at the center of many challenges in system design today. Difficult issues need to …
Generative Deep Learning, 2nd Edition
Generative AI is the hottest topic in tech. This practical book teaches machine learning engineers and …
Software development today is embracing events and streaming data, which optimizes not only how technology interacts …
Foundations of Scalable Systems
In many systems, scalability becomes the primary driver as the user base grows. Attractive features and …