Thinking with Data

Chapter 1. Scoping: Why Before How

Most people start working with data from exactly the wrong end. They begin with a data set, then apply their favorite tools and techniques to it. The result is narrow questions and shallow arguments. Starting with data, without first doing a lot of thinking, without having any structure, is a short road to simple questions and unsurprising results. We don’t want unsurprising—we want knowledge.

As professionals working with data, our domain of expertise has to be the full problem, not merely the columns to combine, transformations to apply, and models to fit. Picking the right techniques has to be secondary to asking the right questions. We have to be proficient in both to make a difference.

To walk the path of creating things of lasting value, we have to understand elements as diverse as the needs of the people we’re working with, the shape that the work will take, the structure of the arguments we make, and the process of what happens after we “finish.” To make that possible, we need to give ourselves space to think. When we have space to think, we can attend to the problem of why and so what before we get tripped up in how. Otherwise, we are likely to spend our time doing the wrong things.

This can be surprisingly challenging. The secret is to have structure that you can think through, rather than working in a vacuum. Structure keeps us from doing the first things to cross our minds. Structure gives us room to think through all the aspects of a ...