Practical A/B Testing

Increasing Confidence in Your Test Results

This section gives you a starting point for the core statistical concepts used to configure an A/B test. It won’t be comprehensive, but it will initiate the ideas and concepts to get you started with configuring an experiment so you can have high confidence in its results.

Configuring your A/B test correctly, such that you can have high confidence in the results, is essential. The more confident you are with your test configuration, the more confident you’ll be in making a decision based on the data from the test.

Calculating Sample Size

The sample size is the number of users exposed to your A/B test. If the sample size is too small, your test results may not accurately represent how your entire user ...