A/B Testing: The Most Powerful Way to Turn Clicks Into Customers

CHAPTER12

Beyond the Page: Non-Website A/B Testing

How to Test Email and Pricing

Up to this point, we've primarily looked at examples where teams test a page's various design elements: forms, images, headlines, and layout. This chapter explores A/B testing beyond the page, in two different domains: email and pricing.

The What and the When: Prezi

Email is one of the most important things you can A/B test. Just as you can use A/B testing to make the elements of your website work better, you can A/B test emails to increase quantifiable success metrics like open rate and click-through rate. And just as you can roll out a website change gradually and gauge response before showing it to all users, it's easy and incredibly advantageous to roll out an email in the same way. First, select a portion of your total mailing list and send them a number of variations. Gauge your success metrics, and then send the email that works best to the remainder of your full list.

It was Halloween of 2011, and David Malpass—marketing analyst at cloud-based presentation tool Prezi—was planning to send out a huge email newsletter blast. He sent his boss, the director of marketing, three candidate subject lines:

1. “New Tricks & Treats”
2. “New features: Templates, Google Image Search”
3. “Create a beautiful Prezi in minutes”

FIGURE 12.1 The open rates for the four Prezi email subject lines.

Source: Prezi

Malpass ...