Behavioral Data Analysis with R and Python

Chapter 9. Stratified Randomization

In the previous chapter, we saw the simplest form of randomization: a customer shows up, and we toss a metaphorical coin or dice. Heads and they see version A, tails and they see version B. The probabilities may be different from 50/50, but they are constant, and independent of the customer characteristics. No “my control group is a bit older than my treatment group, let’s make sure the next Millennial who shows up goes into the control group.” As a consequence, your control and treatment groups are “probabilistically equivalent,” which is statistics’ way of saying that if you kept running your experiment forever, your two groups would have the exact same proportions as your general population. In practice, however, your experimental groups; can end up being quite different from each other. Adding explanatory variables to your final analysis can somewhat compensate for these imbalances, but as we’ll now see, we can do better than that if we know ahead of time who is going to be part of our experiment.

In this chapter, I’ll introduce you to stratified randomization, which will allow us to ensure that our experimental groups are as similar as possible. This starkly increases the explanatory power of an experiment, which is especially useful when you can’t have large sample sizes.

Stratified randomization can be applied to any situation where we have a predetermined list of customers/employees/etc. to build our experimental groups from. Given that ...