O'Reilly logo
live online training icon Live Online training

Testing and Validating Product Ideas with Lean

Understand Product Market Fit and Integrate Customer Feedback

Kylie Castellaw
Maryam Aidini

How can enterprises reap the benefits of lean and its iterative, learning-based approach?

Maybe you’ve heard about lean but always figured it was for different types of projects than yours. Or perhaps you’ve worked on projects that were disasters in the market and want to avoid that outcome in the future. Or you’re looking far ways to end lengthy discussions filled with opposing opinions about how to move forward with a project. It’s time to get on board with lean testing and validating.

You’ll learn about some simple testing techniques to get quick, cheap data that can help us build successful products by build a foundation for testing, customer feedback, and provide support on learning. Topics addressed will include getting feedback from customers and incorporating it into your ideas and understanding how to integrate changes to make the best product possible in future. We’ll discuss how to address stakeholder and team-member concerns with lean experimentation. This is especially important if you often hear things like “not another process/buzzword/thing startups do!” And we’ll help you create a lean experiment of your own.

What are the Big Ideas in this course? - Use the Build Measure Learn approach to craft and carry out targeted experiments to validate and pivot your product ideas - Gain confidence in your product development decision making - Learn to reduce the risk of product failure

Why are those important for what companies need? - Too many products fail too hard after too long in the market. In order to have a successful product, and keep pace with changing business needs, organizations are quickly learning that they need to be nimble in product development. Many product managers also have a hard time making sound decisions based on data in companies that don’t have metrics tracking baked in -- the lean experimentation technique can help streamline the decision-making process by rooting decisions in real customer data faster.

What you'll learn-and how you can apply it

Participants will understand… - Why people are talking about Lean experimentation and how it can help them in their daily role and decision making - The importance of failing fast and what that means - How to design and run lean experiments on their projects

Participants will hear... - A quick overview of the lean approach as it pertains to product development - Multiple examples of how to validate product assumptions using a simple Lean experimentation method - Lessons the instructors have learned about the advantages of using lean experimentation

Participants will be able to… - Identify latent assumptions that they and their teams hold about the success of a given project or product - Identify and create small experiments to validate their product assumptions - Review experiment results and decide how to pivot their product as needed

Common Misunderstandings: - Lean experimentation is time-consuming and expensive - Releasing your product before it’s fully formed leaves you vulnerable to copycats - You need a lot of data and test users to have any experiment be meaningful

This training course is for you because...

  • You are a product manager tasked with building a successful product and want to get a clearer picture of how that can happen
  • You are a project contributor (developer, designer, etc.) with experience creating products/software and you want to make sure you are building the right thing
  • You have a great idea for a startup or new product at your company but aren’t sure how to tell if it will be viable in the market.
  • You are an executive that wants to expand the portfolio and to be able to identify which initiatives to invest on, you want product teams to give you more data to empower you to make the right decision

Outcomes - You can describe the Lean experimentation cycle - You can identify specific areas of their work where Lean experimentation could be helpful - You can identify at least three potential obstacles to adopting Lean experimentation and can name potential ways to address them


  • A familiarity with basic terminology in the domain of the creation of digital products
  • Understand the basic concept of Agile at a high level (see Reading Assignments)
  • A basic understanding of the Lean approach (or previously taken the Introduction to Lean course)

Pre-work instructions: Think about a past project of yours (or one that you’re aware of from a different company) that flopped in the market. Why do you think that was? List at least 4 reasons for the flop. We will begin by discussing your reflections.

Reading Assignments - Read The Lean Enterprise, Chapter 1 - Read The Lean Entrepreneur, Chapter 2 - Read the article “The Discipline of Business Experimentation” from Harvard Business Review’s 10 Must Reads 2016

About your instructor

  • Kylie Castellaw has always loved science and art, and came to the Lean practice as a UX designer/researcher and cognitive psychologist. She is currently a lead consultant at ThoughtWorks San Francisco, where she coaches Fortune 500 and nonprofit teams to work leanly in portfolio planning, product development, concept ideation, and experience design.

  • Maryam Aidini started her career as a software engineer, and has worked with many teams over the years to deliver software products using Lean and agile practices. She is currently a lead at ThoughtWorks London and spends her time coaching enterprise clients in the portfolio management, product definition, and software delivery phases of their product life cycles.


The timeframes are only estimates and may vary according to how the class is progressing

Part 1: Lean experimentation set up

Segment 1: What is Lean experimentation?<br> This segment focuses on giving context to Lean learning cycles in general and Lean experimentation in particular. - Summary of what Lean experimentation is - Lean experimentation journey - Advantages of Lean experimentation mindset - Lean experimentation success stories

Exercise: Poll about past experimentation experience (Have you been in a situation where the project failed?)

Segment 2: Uncovering assumptions<br> This segment demonstrates the importance of identifying assumptions and specific tactics to uncover them. A sample product case study will be reviewed. - What are assumptions and their importance - Techniques to uncover assumptions - Demonstration example

Exercise: (10 mins) _ In this exercise, participants will complete an assignment to pinpoint some assumptions about the example product in Slack. Presenters will share sample answers._


Part 2: Experiments revealed

Segment 3: Creating experiments to test your assumptions<br> This segment will demonstrate some techniques to design and conduct experiments to test your assumptions. - Ways to test different types of assumptions - Ideas for experiments - Tips and tricks to running successful experiments - Demonstration example

*Exercise: In this exercise, participants will complete an assignment to design an experiment for the example product in Slack. Presenters will share sample answers.


Segment 4: Analyzing experiment results and defining next steps<br> This segment will focus on how to analyze experimental results to validate assumptions, as well as a practical framework to determining next steps - Techniques to analyze experiment results - Potential paths forward, including pivots - Demonstration example

Conclusion - Review Lean experimentation cycle - Common questions - Addressing concerns with Lean experimentation - For the road: worksheet to plan an experiment in your context