Hands-On Q-Learning with Python

Upper confidence bound

Upper confidence bound is another action selection approach we can take.

The goal of upper confidence bound is to make sure we don't continue to take random actions we've already taken and that have given us poor results. It prioritizes action spaces with high uncertainty (spaces we haven't explored yet at all get priority over spaces we have explored). Essentially, we are putting an upper limit on the utility (value) of a potential action.

In a Bayesian UCB strategy, we estimate a prior probability distribution for the reward (such as Gaussian). In other strategies we don't assume any prior distributions and use other types of estimate instead (such as Hoeffding's inequality).

The following distributions are three ...