Hands-On Q-Learning with Python

Thompson sampling

Essentially, Thompson sampling has us believing what the prior probability distribution is and updating it every time we get new information about the environment. Eventually, our updated belief will coincide with the true probability distribution. This approach is fundamentally Bayesian. This is because it treats the probability distribution as our current lack of knowledge about the environment and updates it according to the new information we get.

In Thompson sampling, we are working with a contextual (state-based) bandit framework. The agent receives information about its state, the actions available to it, and the current Q-values of those actions.

We start with a prior likelihood distribution on each action A based ...