Big Data Now: 2014 Edition

Chapter 3. The Convergence of Cheap Sensors, Fast Networks, and Distributed Computing

One of the great drivers in the development of big data technologies was the explosion of data inflows to central processing points, and the increasing demand for outputs from those processing points—Google’s servers, or Twitter’s, or any number of other server-based technologies. This shows no signs of letting up; on the contrary, we are creating new tools (and toys) that create data every day, with accelerometers, cameras, GPS units, and more.

Ben Lorica kicks off this chapter with a review of tools for dealing with this ever-rising flood of data. Then Max Shron gives a data scientist’s perspective on the world of hardware data: what unique upportunities and constraints does data produced by things that are 99% machine, 1% computer bring? How to create value from flows of data is the next topic, covered by Mike Barlow, and then Andy Oram covers a specific use case: smarter buildings. Finally, in a brief coda, Alistair Croll looks ahead to see even more distribution of computing, with more independence of devices and computation at the edges of the network.

Expanding Options for Mining Streaming Data

New tools make it easier for companies to process and mine streaming data sources

Stream processing was in the minds of a few people that I ran into over the past week. A combination of new systems, deployment tools, and enhancements to existing frameworks, are behind the recent chatter. Through ...