Growth Engineering

CHAPTER 5What Are Experiments?

Walk into any product team’s planning meeting and you’ll hear opinions. Some people are convinced that adding a new feature will boost engagement. Others argue that the sign-up flow is too long. Someone else swears that changing the button color will improve conversions. All these ideas may be valid, but until they’re tested, they’re just opinions.

Experimentation transforms opinions into evidence. It is the discipline that helps teams decide which ideas are worth building, which should be discarded, and which deserve another iteration. Without it, product development can feel like driving in fog. You’re moving forward, but you’re not sure whether you’re headed toward the highway or a ditch. With it, you have headlights, road signs, and a global positioning system (GPS) telling you which direction works.

At its core, experimentation is a mindset. It’s about embracing uncertainty and seeing every change to your product as a testable hypothesis rather than as a guaranteed success. This mindset is not natural for many organizations. Psychologists have shown that people consistently overestimate the accuracy of their own knowledge and predictions. This tendency, known as overconfidence bias, shows up everywhere from business decisions to everyday judgments.¹ In practice, that means executives often rely on experience, engineers trust their intuition, and designers lean on their instincts about what looks and feels right. But growth engineering demands ...