Improving Product with A/B Testing
Using A/B testing and experimentation to drive business and improve customer experience
This live training course will help you understand the WHY behind A/B testing and how organizations use it to drive innovation, optimization, and growth. Increasingly companies are using A/B testing, and looking for developers, designers, data scientists, and product managers with experience working with it. Whether you have never implemented tests before, or you are on a team implementing testing, you will learn why companies choose to A/B test, what makes for impactful tests, and how to tie tests to company goals. All attendees will walk away understanding the basics of experimentation, why organizations do it and how it impacts goals, a high level of how companies implement A/B testing, and how to create relevant and impactful tests.
What you'll learn-and how you can apply it
- What A/B testing is and isn’t
- Why companies choose to A/B test
- What roles are involved with experimentation and testing
- How to create impactful tests
- Creating tests that align with your organization’s goals
- Define success and failure for tests
- Bonus: How to talk about experimentation and testing in interviews
This training course is for you because...
- You are a developer, designer, data scientist, or product manager interested in A/B testing and want to learn how it works.
- You are a developer, designer, data scientist, or product manager currently doing A/B testing and want to improve your current experimentation and understanding of the why behind testing and context for testing.
- You are a developer, designer, data scientist, or product manager and want to know how to discuss your A/B testing experience in interviews, or how to discuss it in general if you don’t already have experience.
- Basic understanding of A/B testing (no experience required)
- Close applications that automatically sync like Dropbox, Google Drive, etc
- Book: Experimentation Works: The Surprising Power of Business Experiments by Stefan H. Thomke https://www.oreilly.com/library/view/experimentation-works/9781633697119/ (Optional)
- Book Understanding Experimentation Platforms by Adil Aijaz, Trevor Stuart, Henry Jewkes https://www.oreilly.com/library/view/understanding-experimentation-platforms/9781492038139/ (Optional, best for developers and data scientists)
- Book: Experimentation, Validation, and Uncertainty Analysis for Engineers, 4th Edition by W. Glenn Steele, Hugh W. Coleman https://www.oreilly.com/library/view/experimentation-validation-and/9781119417514/
About your instructor
Anne is a Product Manager and Product Coach, and formerly the Lead Product Manager at OpenLaw, blockchain-based protocol and markup language to prepare, manage, and execute smart legal agreements. Her focus is on growth and emerging technologies with a degree in engineering from the University of Michigan. She is passionate about the practical human aspects of technology and building AI and blockchain products rooted in the realities of the human experience. She is also an Emerging Tech Correspondent for Tech 2025, a platform and community for learning about, and discussing emerging technologies. Additionally, she serves on the Rutgers University Big Data Certificate program Advisory Board and has guest lectured at Morgan State University and the University of Montreal on blockchain.
The timeframes are only estimates and may vary according to how the class is progressing
Section 1: Basics of A/B Testing (length 30 min)
- What is A/B testing
- Why do we do A/B tests
- What is A/B testing good for
Break and Q&A
Section 2: A/B Testing Meets Reality In a Company (60 min)
- Defining goals and success for an experimentation program in your organization
- How experimentation teams work and what roles are on these teams
- A/B testing in different company stages
- Experimentation for growth teams vs experimentation for optimization
Break and Q&A
Section 3: Building Experiments (60 min)
- What makes a good experiment vs a bad experiment
- A/B hypothesis exercise
- Measuring success for experimentation (quantitative and qualitative metrics)
- Statistics behind A/B testing
Section 4: Wrap Up (15 min)