Hands-On Reinforcement Learning with Python

The Thompson sampling algorithm

Thompson sampling (TS) is another popularly used algorithm to overcome the exploration-exploitation dilemma. It is a probabilistic algorithm and is based on a prior distribution. The strategy behind TS is very simple: first, we calculate prior on the mean rewards for each of the k arms, that is, we take some n samples from each of the k arms and calculate k distributions. These initial distributions will not be the same as the true distribution, so we call it prior distribution:

Since we have Bernoulli rewards, we use beta distribution for calculating the prior. The value of beta distribution [alpha, beta] ...