Avoiding Data Pitfalls

Chapter SixPitfall 5: Analytical Aberrations

“Data is a tool for enhancing intuition.”

—Hilary Mason

How We Analyze Data

What is the purpose of collecting data? People gather and store data for at least three different reasons that I can discern. One reason is that they want to build an arsenal of evidence with which to prove a point or defend an agenda that they already had to begin with. This path is problematic for obvious reasons, and yet we all find ourselves traveling on it from time to time.

Another reason people collect data is that they want to feed it into an artificial intelligence algorithm to automate some process or carry out some task. This purpose involves a set of activities that I haven't really included in this book. But it, too, is fraught with pitfall after pitfall, on which I hope to write at some point in the future.

A third reason is that they might be collecting data in order to compile information to help them better understand their situation, to answer questions they have in their mind, and to unearth new questions that they didn't think to ask.

This last purpose is what we call data analysis, or analytics.

Pitfall 5A: The Intuition/Analysis False Dichotomy

Some years ago I saw a television commercial for Business Intelligence (BI) software in which a customer being interviewed had the following to say:

We used to use intuition; now we use analytics.

In other words, we're being asked to believe that the business owner was able to make progress ...