Every experiment has strengths and weaknesses. Some are quick and cheap but produce less reliable evidence. Some produce more reliable evidence but require more time and money to execute.
Consider cost, data reliability, and time required when you design your mix of experiments. As a rule of thumb, start cheap when uncertainty is high and increase your spending on experiments with increasing certainty.
Select a series of tests by drawing from our experiment library or by using your imagination to invent new experiments. Keep two things in mind when you compose your mix:
What customers say and do are two different things.
Use experiments that provide verbal evidence from customers as a starting point. Get customers to perform actions and engage them (e.g., interact with a prototype) to produce stronger evidence based on what they do, not what they say.
Customers behave differently when you are there or when you are not.
During direct personal contact with customers, you can learn why they do or say something and get their input on how to improve your value proposition. However, your presence might lead them to behave differently than if you weren’t there.
In an indirect observation ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month, and much more.
