Designing Big Data Platforms

2A Bird's Eye View on Big Data

After reading this chapter, you should be able to

• Learn chronological information about Big Data processing
• List qualities that characterize Big Data
• List components of Big Data platforms
• Describe uses cases for Big Data

Development of Big Data platforms has spanned for over two decades. To provide an overview of its evolution, this chapter will present the qualities of Big Data, components of Big Data platform, and use cases of Big Data.

2.1 A Bit of History

Computing has advanced drastically in the past two decades, from network to data storage, with significant improvements. Despite the rapid changes, the definition of Big Data remains relevant. This section presents the evolution of Big Data chronologically.

2.1.1 Early Uses of Big Data Term

The term Big Data was used and described by Cox and Ellsworth (1997), who presented two ideas: Big Data collections and Big Data objects. Big Data collections are streamed by remote sensors as well as satellites. The challenge is pretty similar to today's Big Data where data is unstructured and has different data sources. Big Data objects are produced from large‐scale simulations of computational dynamics and weather modeling. The combined problem of the Big Data object and Big Data collections are again comparable to today's Big Data challenges where data is too large for memory and disk to fit a single machine.

In his presentations regarding Big Data, Mashey (1998) noted that the need for storage ...