O'Reilly logo

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Improve the outcome of your data experiments with A-B testing

Book Description

Data scientists are faced with the need to conduct continual experiments, particularly regarding user interface and product marketing. Designing experiments is a cornerstone of the practice of statistics, with clear application to data science. In this lesson, you’ll learn about A-B testing and hypothesis, or significance tests—critical aspects of experimental design for data science.

What you’ll learn—and how you can apply it

You will learn the central concepts of A-B testing, understand its role in designing and conducting data science experiments, and the characteristics of a proper A-B test. Through a series of sample tests, you’ll learn how to interpret results, and apply that insight to your analysis of the data. Since A-B tests are typically constructed with a hypothesis in mind, you’ll also learn how to conduct various hypothesis, or significance tests, enabling you to avoid misinterpreting randomness.

This lesson is for you because

You are a data scientist or analyst working with data, and want to gain beginner-level knowledge of key statistical concepts to improve the design, and outcome of your experimental tests with data.


  • Basic familiarity with coding in R

Materials or downloads needed:

  • n/a