Dive Into Data Science

4 A/B Testing

In the preceding chapter, we discussed the scientific practice of observing two groups and making quantitative judgments about how they relate to each other. But scientists (including data scientists) do more than just observe preexisting differences. A huge part of science consists of creating differences experimentally and then drawing conclusions. In this chapter, we’ll discuss how to conduct those kinds of experiments in business.

We’ll start by discussing the need for experimentation and our motivations for testing. We’ll cover how to properly set up experiments, including the need for randomization. Next, we’ll detail ...