Real-Time Big Data Analytics: Emerging Architecture

Chapter 3. How Real Is Real Time?

Here’s another complication: The meaning of “real time” can vary depending on the context in which it is used.

“In the same sense that there really is no such thing as truly unstructured data, there’s no such thing as real time. There’s only near-real time,” says John Akred, a senior manager within the data domain of Accenture’s Emerging Technology Innovations group. “Typically when we’re talking about real-time or near real-time systems, what we mean is architectures that allow you to respond to data as you receive it without necessarily persisting it to a database first.”

In other words, real-time denotes the ability to process data as it arrives, rather than storing the data and retrieving it at some point in the future. That’s the primary significance of the term — real-time means that you’re processing data in the present, rather than in the future.

But “the present” also has different meanings to different users. From the perspective of an online merchant, “the present” means the attention span of a potential customer. If the processing time of a transaction exceeds the customer’s attention span, the merchant doesn’t consider it real time.

From the perspective of an options trader, however, real time means milliseconds. From the perspective of a guided missile, real time means microseconds.

For most data analysts, real time means “pretty fast” at the data layer and “very fast” at the decision layer. “Real time is for robots,” says Joe Hellerstein, ...