O'Reilly logo
live online training icon Live Online training

Improving Product with A/B Testing

Using A/B testing and experimentation to drive business and improve customer experience

Topic: Data
Anne Griffin

This live training course will help you understand the WHY behind A/B testing and how organizations use it to drive innovation, optimization, and growth. Increasingly companies are using A/B testing, and looking for developers, designers, data scientists, and product managers with experience working with it. Whether you have never implemented tests before, or you are on a team implementing testing, you will learn why companies choose to A/B test, what makes for impactful tests, and how to tie tests to company goals. All attendees will walk away understanding the basics of experimentation, why organizations do it and how it impacts goals, a high level of how companies implement A/B testing, and how to create relevant and impactful tests.

What you'll learn-and how you can apply it

  • What A/B testing is and isn’t
  • Why companies choose to A/B test
  • What roles are involved with experimentation and testing
  • How to create impactful tests
  • Creating tests that align with your organization’s goals
  • Define success and failure for tests
  • Bonus: How to talk about experimentation and testing in interviews

This training course is for you because...

  • You are a developer, designer, data scientist, or product manager interested in A/B testing and want to learn how it works.
  • You are a developer, designer, data scientist, or product manager currently doing A/B testing and want to improve your current experimentation and understanding of the why behind testing and context for testing.
  • You are a developer, designer, data scientist, or product manager and want to know how to discuss your A/B testing experience in interviews, or how to discuss it in general if you don’t already have experience.


  • Basic understanding of A/B testing (no experience required)

Course Set-up

  • Laptop
  • Close applications that automatically sync like Dropbox, Google Drive, etc

Recommended Follow-up

About your instructor

  • Anne is a Product Manager and Product Coach, and formerly the Lead Product Manager at OpenLaw, blockchain-based protocol and markup language to prepare, manage, and execute smart legal agreements. Her focus is on growth and emerging technologies with a degree in engineering from the University of Michigan. She is passionate about the practical human aspects of technology and building AI and blockchain products rooted in the realities of the human experience. She is also an Emerging Tech Correspondent for Tech 2025, a platform and community for learning about, and discussing emerging technologies. Additionally, she serves on the Rutgers University Big Data Certificate program Advisory Board and has guest lectured at Morgan State University and the University of Montreal on blockchain.


The timeframes are only estimates and may vary according to how the class is progressing

Section 1: Basics of A/B Testing (length 30 min)

  • What is A/B testing
  • Why do we do A/B tests
  • What is A/B testing good for

Break and Q&A

Section 2: A/B Testing Meets Reality In a Company (60 min)

  • Defining goals and success for an experimentation program in your organization
  • How experimentation teams work and what roles are on these teams
  • A/B testing in different company stages
  • Experimentation for growth teams vs experimentation for optimization

Break and Q&A

Section 3: Building Experiments (60 min)

  • What makes a good experiment vs a bad experiment
  • A/B hypothesis exercise
  • Measuring success for experimentation (quantitative and qualitative metrics)
  • Statistics behind A/B testing

Section 4: Wrap Up (15 min)

  • Recap
  • Q&A