Video description
Deciding whether or not to launch a new product or feature is a resource management bet for any Internet business. Conducting rigorous online A/B tests flattens the risk. Drawing on her experience at Airbnb, data scientist Lisa Qian offers a practical ten-step guide to designing and executing statistically sound A/B tests.
- Discover best practices for defining test goals and hypotheses
- Learn to identify controls, treatments, key metrics, and data collection needs
- Understand the role of appropriate logging in data collection
- Determine how to frame your tests (size of difference detection, visitor sample size, etc.)
- Master the importance of testing for systematic biases
- Run power tests to determine how much data to collect
- Learn how experimenting on logged out users can introduce bias
- Understand when cannibalization is an issue and how to deal with it
- Review accepted A/B testing tools (Google Analytics, Vanity, Unbounce, among others)
Lisa Qian focuses on search and discovery at Airbnb. She has a PhD in Applied Physics from Stanford University.
Publisher resources
Table of contents
- Overview of the Course 00:02:15
- Why Should You Run A/B Tests? 00:04:18
- The 10 Steps and An Overview of Case Studies 00:03:01
- Case Study 1: Red vs. Green Button 00:22:16
- Case Study 2: Testing a New Landing Page 00:16:11
- Case Study 3: Price Recommendations on an Online Marketplace 00:11:46
- Summary: Setting Up an A/B Test 00:05:16
- Some of Your Options 00:04:53
- Scaling A/B Testing and Developing a Culture of Experimentation 00:06:53
Product information
- Title: A/B Testing, A Data Science Perspective
- Author(s):
- Release date: September 2015
- Publisher(s): O'Reilly Media, Inc.
- ISBN: 9781491934777
You might also like
book
Bayesian Statistics the Fun Way
Probability and statistics are increasingly important in a huge range of professions. But many people use …
book
Practical Time Series Analysis
Time series data analysis is increasingly important due to the massive production of such data through …
book
Designing with Data
On the surface, design practices and data science may not seem like obvious partners. But these …
book
Think Stats, 2nd Edition
If you know how to program, you have the skills to turn data into knowledge, using …