The Subtle Sources of Sampling Bias Hiding in Your Data

Plummeting data acquisition costs have contributed to a surge in business analytics. But more data doesn’t inherently remove sampling bias — and in some cases, it could make it worse.

When a group of Boston College students started an analytics project using data about UFO sightings, they thought they’d learn something about visits from spaceships and alien creatures — such as how weather and movie releases influence sightings. The Economist had done something similar, finding that most UFO reports are made during what it called "drinking hours" (5 to 11 p.m.), when people could be “nursing their fourth beer” — a possible connection that the publication dubbed ...

Get Strategies for AI and Cognitive Tools now with O’Reilly online learning.

O’Reilly members experience live online training, plus books, videos, and digital content from 200+ publishers.