April 2016
Intermediate to advanced
325 pages
9h 24m
English
Content preview from Designed for Use, 2nd EditionBecome an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
Start your free trial
Interpreting the Results
Most A/B tools interpret the results for you; quite often, you can see early trends indicating which design performs best. But be careful: humans are pattern seekers, and it’s easy to see patterns where there are none. One thing to keep in mind here is statistical significance. You need to have some idea of how big the probability is that the results you’re seeing have occurred because of random chance.
The company User Effect[173] has a simple calculator[174] that reveals whether the results of an A/B test with two designs are statistically significant. If they’re not, the calculator also tells you how many more visitors you need to get a significant result.
If you have more than two designs in your A/B test, you can ...