Chapter 10

Special Considerations in Big Data Analysis

Outline

Those who ignore Statistics are condemned to reinvent it.

Brad Efron

Background

Big Data statistics are plagued by several intrinsic flaws. When the amount of data is sufficiently large, you can find almost anything you seek lurking somewhere within; the found observations may have statistical significance without having any significance in reality. Also, whenever you select a subset of data from an enormous collection, you may have no way of knowing the relevance of the data that ...

Get Principles of Big Data now with O’Reilly online learning.

O’Reilly members experience live online training, plus books, videos, and digital content from 200+ publishers.