Hands-On Q-Learning with Python

Testing methodologies

Effectively, A/B testing is running a period of pure exploration (using the two options equally to compare their performance), followed by a period of pure exploitation (using the winning option all of the time).

With the multi-armed bandit approach, we are mixing our exploration and our exploitation phases and maximizing our output based on the knowledge we currently have of our action space (the ads we have to choose from).

In other words, we're choosing the best possible option we have now, rather than waiting until the end of the trial to choose the option that won out most often.

The following diagram shows how this works. In A/B testing, we spend half the time doing pure exploration, alternating between A and ...