Designing for Behavior Change, 2nd Edition

Chapter 13. Determining Impact with A/B Tests and Experiments

You should take the approach that you’re wrong. Your goal is to be less wrong.

Elon Musk

Opower, an energy-efficiency and customer engagement unit owned by Oracle Utilities, has run some of the largest experiments in the world about how products can change behavior. Millions of people have participated in their studies simply by opening a letter from their utility company or fiddling with their thermostat.

Opower is best known for delivering monthly reports to utility customers that show them how their energy usage stacks up against their (anonymous) neighbors. It’s a well-studied technique in social psychology called peer comparisons, which we discussed a bit in Chapter 9. Figure 13-1 shows an example of one of its comparisons.

A host of government, private, and academic publications have shown that Opower’s simple comparisons help consumers cut their energy bills by roughly 2% on average.¹ That may seem small, but it adds up to over 2.6 terawatt-hours of electricity—enough to power 300,000 homes for a year, or roughly $300 million in consumer savings on energy bills.²

Figure 13-1. An Opower energy report, comparing the reader’s home heating usage to that of their neighbors

Opower has repeatedly run experiments ...