Real-Time Analytics: Techniques to Analyze and Visualize Streaming Data

Chapter 2Designing Real-Time Streaming Architectures

By their nature, real-time architectures are layered systems that rely on several loosely coupled systems to achieve their goals. There are a variety of reasons for this structure, from maintaining the high availability of the system in an unreliable world to service requirements and managing the cost structure of the architecture itself.

The remainder of the first section of this book introduces software, frameworks and methods for dealing with the various elements of these architectures. This chapter serves as the blueprint and foundation for the architecture. First, the various components of the architecture are introduced. These usually, but not always, correspond to separate machine instances (physical or virtual). After these components have been introduced, they can be discussed in the context of the primary features of a real-time architecture: high availability, low latency, and horizontal scalability.

This chapter also spends some time discussing the languages used to build real-time architectures. It is assumed that the reader will have some familiarity with the languages used for the examples in this book: Java and JavaScript. Many of the software packages in this book are implemented using the Java Virtual Machine, though not necessarily Java itself. JavaScript, of course, is the lingua franca of the web and is used extensively in the later sections of this book to implement the interfaces to the data.

Finally, ...