7. On Analysis
Trusting your findings
Understanding the results of your experiments can be a complicated affair. Luckily, there are a few simple methods to make sense of all the data your experiments accumulate and give you confidence in your findings.
Statistical significance
In your experiments, you attempt to hold all variables constant except the thing you want to investigate. The reality, of course, is that many variables are not constant. You’re comparing similar groups, but those groups themselves will have differences that introduce noise. That’s where statistical significance comes in (Figure 7.1).
Figure 7.1. Statistical significance ...
Get Experiment!: Website conversion rate optimization with A/B and multivariate testing now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
