Principles of Big Data

Chapter 8

Simple but Powerful Big Data Techniques

Outline

Background

Look at the Data

Data Range

Denominator

Frequency Distributions

Mean and Standard Deviation

Estimation-Only Analyses

Use Case: Watching Data Trends with Google Ngrams

Use Case: Estimating Movie Preferences

Only theory can tell us what to measure and how to interpret it.

Albert Einstein

Background

It may come as something of a shock, but most Big Data analysis projects can be done without the aid of statistical or analytical software. A good deal of the value in any Big Data resource can be attained simply by looking at the data and thinking about what you’ve seen.

It is said that we humans have a long-term memory capacity in the petabyte range and that we process many thousands ...