Data Science: The Hard Parts

Chapter 15. Incrementality: The Holy Grail of Data Science?

In the past I’ve argued that incrementality is the holy grail of data science. This statement depends critically on the hypothesis that I’ve maintained throughout: that data science creates value by improving a company’s decision-making capabilities. This chapter expands on this topic, but most importantly, I will present some techniques that should build some basic intuitions that will become handy if and when you decide to delve deeper. As usual, the topic is worthy of a book-length treatment, so I will provide several references at the end of this chapter.

Defining Incrementality

Incrementality is just another name for causal inference applied to decision-making analytics. If you recall from Figure 14-1, a typical decision comprises an action or lever, and an outcome that depends on the underlying uncertainty. If the lever improves the outcome, and you’re able to isolate any other factors that might explain the change, you can say (with some degree of confidence) that it was incremental. For later reference, the action is also known as the treatment, following the more classical medical literature of controlled experiments, where some patients receive a treatment, and the remaining control group receives a placebo.

Causality is commonly defined by use of counterfactuals. As opposed to facts—something that we observe—counterfactuals attempt to provide an answer to the question: what if I had followed a different course ...