O'Reilly logo

Quantifying the User Experience by James R Lewis, Jeff Sauro

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Chapter 5

Is There a Statistical Difference between Designs?

Introduction

Many researchers first realize the need for statistics when they have to compare two designs or products in an A/B test or competitive analysis. When stakes are high (or subject to scrutiny), just providing descriptive statistics and declaring one design better is insufficient. What is needed is to determine whether the difference between designs (e.g., among conversion rates, task times, or ratings) is greater than what we’d expect from chance. This chapter is all about determining whether a difference is statistically significant and how large or small of a difference likely exists in the untested population.

Comparing Two Means (Rating Scales and Task Times)

A central ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required