CHAPTER 3
How to Experiment Online
It doesn’t make any difference how beautiful your guess is, it doesn’t make any difference how smart you are, who made the guess, or what his name is. If it disagrees with experiment, it’s wrong. That’s all there is to it.
—RICHARD FEYNMAN, PHYSICIST, TEACHER, AND STORYTELLER
In 2012, a Microsoft employee working on its Bing search engine had an idea about changing the way it displayed ad headlines.1 The idea seemed trivial: add some of the ad’s subtext to the headline to make it longer (see figure 3-1). Developing the change wouldn’t require much effort—just a few days of an engineer’s time—but it was one of hundreds of ideas proposed, and the program managers deemed it a low priority. So it languished for ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access