Data Management at Scale

Chapter 5. Event and Response Management: The Streaming Architecture

In this chapter we will discuss the Streaming Architecture, which is an event-driven architecture. It is also the most complex because it overlaps with both the RDS and API Architectures. We’ll look at things like asynchronous communication, event-driven architectures, and technologies such as Kafka, consistency models, event types, and more. By the end of this chapter you’ll have a good understanding of what event-driven architectures can bring to your organization.

Introducing the Streaming Architecture

The Streaming Architecture borrows its name from streaming data or event stream processing. These are continuous real-time processing and analyzing of generated data from various sources. This pattern is also called by many other names (the subtle differences between these names will be explained in “Streaming analytics services”): event processing, complex event processing (CEP), real-time analytics, streaming analytics, and real-time streaming analytics.

Real-time streaming data provides a competitive advantage: quicker responsiveness leads to higher customer satisfaction. Faster results make insights more relevant. Instead of waiting minutes or hours, you can react immediately. Fraud can be detected the moment it happens. You can see where your web visitors are coming from, interact with them, and improve their customer experience as they stay on your website. Data delivered in a stream of events can be ...