Chapter 14

Exploring the Evolution of Big Data Technologies

Stephen Bonner; Ibad Kureshi; John Brennan; Georgios Theodoropoulos    Institute of Advanced Research Computing, Durham University, United Kingdom

Abstract

This chapter explores the rise of “big data” and the computational strategies, both hardware and software, that have evolved to deal with this paradigm. Starting with the concept of data-intensive computing, the different facets of data processing like Map/Reduce, Machine Learning, and Streaming data are explored. The evolution of different frameworks such as Hadoop and Spark are outlined and an assessment of the modular offerings within the frameworks is compared with a detailed analysis of the different functionalities and ...

Get Software Architecture for Big Data and the Cloud now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.