A/B Testing: The Most Powerful Way to Turn Clicks Into Customers

CHAPTER2

What to Test

Optimization in Five Steps

The hardest part of A/B testing is determining what to test in the first place. Having worked with thousands of customers who do A/B testing every day, one of the most common questions we hear is, “Where do I begin?”

A mistake that some companies make is to start moving a bunch of levers around without clear planning upfront for what they're trying to optimize—and what will be impacted by those changes. It's tempting to just dive in and start changing parts of your homepage, or your product page, or your checkout page, without truly understanding the value that it's generating (or not generating) for your business.

Instead, we advise a purposeful and deliberate five-step process:

Step One: Define success

Step Two: Identify bottlenecks

Step Three: Construct a hypothesis

Step Four: Prioritize

Step Five: Test

This process begins with the most important question of all: What is the purpose of your site?

Step One: Define Success

Before you can determine which of your test's variations is the winner, you have to first decide how you're keeping score. To start A/B testing successfully, you need to answer a specific question: What is your website for? If you could make your website do one thing better, what would it do?

If the answer to that question isn't completely clear to you, there's a trick that might help. Imagine the following dialogue:

ALICE: “What do you want to achieve with A/B testing?”

BOB: “We don't know. We don't know ...