Bandit Algorithms for Website Optimization

Chapter 1. Two Characters: Exploration and Exploitation

To set the stage for this book, I’m going to tell you a short story about a web developer, Deborah Knull, who ran a small web business that provided most of her income. Deb Knull’s story will introduce the core concepts that come up when studying bandit algorithms, which are called exploration and exploitation. To make those ideas concrete, I’m going to associate them with two types of people: a scientist who explores and a businessman who exploits. My hope is that these two characters will help you understand why you need to find a way to balance the desires of both of these types of people in order to build a better website.

The Scientist and the Businessman

One Sunday morning, a young web enterpreneur, Deb Knull, came to suspect that changing the primary color of her site’s logo would make her site’s users feel more comfortable. Perhaps more importantly, she thought that making her customers feel more comfortable would make them buy more of the products her site was selling.

But Deb Knull worried that a new color could potentially disorient users and make them feel less comfortable. If that were true, her clever idea to increase sales might actually make her users buy fewer products instead. Unsure which of her instincts to trust, she asked for advice from two of her friends: Cynthia, a scientist, and Bob, a businessman.

Cynthia the Scientist

Cynthia, the scientist, loved Deb’s proposed logo change. Excited by the opportunity ...