Designed for Use, 2nd Edition

Running the Test

When running A/B tests, you want to make sure that individual users don’t switch between the two different designs. (Seeing one version of the design and then, after clicking a link or restarting the application, suddenly seeing the other version would be really confusing.) To do that, you should partition your users into different groups that consistently see the same design.

While the test is running, you’ll need a way of collecting the results. This is simple when you’re dealing with a website (since you can “see” what each user is doing), but it can be harder when dealing with a desktop application.

You also need to inform users that you’re receiving usage data and explain exactly what you’re going to send back and why. The ...