Real-Time Big Data Analytics: Emerging Architecture

Chapter 7. Part of a Larger Trend

The push toward real-time big data analytics is part of a much larger trend in which the machines we create act less like machines and more like human beings, says Dhiraj Rajaram, Founder and CEO of Mu-Sigma, a provider of decision sciences and analytics solutions.

“Today, most of our technology infrastructure is not designed for real time,” says Rajaram, who worked as a strategy consultant at Booz Allen Hamilton and Pricewaterhouse Coopers before launching Mu Sigma. “Our legacy systems are geared for batch processing. We store data in a central location and when we want a piece of information, we have to find it, retrieve it and process it. That’s the way most systems work. But that isn’t the way the human mind works. Human memory is more like flash memory. We have lots of specific knowledge that’s already mapped — that’s why we can react and respond much more quickly than most of our machines. Our intelligence is distributed, not highly centralized, so more of it resides at the edge. That means we can find it and retrieve it quicker. Real time is a step toward building machines that respond to problems the way people do.”

As information technology systems become less monolithic and more distributed, real-time big data analytics will become less exotic and more commonplace. The various technologies of data science will be industrialized, costs will fall and eventually real-time analytics will become a commodity.

At that point, the focus will shift ...