Practical Web Analytics for User Experience

Chapter 13

A/B Testing

Contents

Introduction

Designing An Experiment

Select a Page That You Wish to Improve

Determine a Metric for Judging Improvement

Design One or More Alternatives

Tracking Code

Tools

Google Content Experiments

Specialized Tools

Estimating the Length of a Test

Monitoring and “Winning”

Ending a Test Early

Key Takeaways

Introduction

A/B testing is when you take two competing designs for a page, select a single metric for determining success (typically, the conversion rate), and send users to both pages until there is a statistically significant difference in the conversion rate. One could compare a brand new design against an existing design, or compare two new pages.

This technique allows you to confidently make decisions about ...