Practical Java Programming for IoT, AI, and Blockchain

CHAPTER 11Java Programming for Big Data Applications

Data is the new oil.

—Clive Humby

• 11.1 What Is Big Data?
• 11.2 Sources of Big Data
• 11.3 The Three Vs of Big Data
• 11.4 Benefits of Big Data
• 11.5 What Is Hadoop?
• 11.6 Hadoop Key Components
• 11.7 Implementing Hadoop on a Raspberry Pi Cluster
• 11.8 Java Hadoop Example
• 11.9 Summary
• 11.10 Chapter Review Questions

11.1 What Is Big Data?

Data is important for our lives, and we are using data all the time. For example, experimental data saved in a file, student record or employee record saved in a database, sales figures saved in a spreadsheet, as well as common Word, Excel, PowerPoint files, sound files, and movie files, are all traditional data files, which can be stored, analyzed, and displayed using a standard personal computer. The sizes of traditional data files range from kilobytes (KB, 2¹⁰ bytes), megabytes (MB, 2²⁰ bytes) to gigabytes (GB, 2³⁰ bytes), and sometimes to terabytes (TB, 2⁴⁰ bytes). However, with the rapid expansion of the Internet and the increase in mobile users, data can be measured with sizes larger than petabytes, 2⁵⁰ bytes. These data stores are called big data. Big data is too large and too complicated to be stored and analyzed using traditional computer hardware and software.

According to Wikipedia, the world's technological per-capita data capacity has doubled roughly every 40 months since the 1980s. As of 2012, about 2.5 exabytes (2.5×10¹⁸ bytes) of data were generated every day. According to an International ...