Hands-On Q-Learning with Python

Bandit regret

The overall loss we incur due to picking non-optimal arms throughout our series of trials is called regret. We treat regret as a variable we want to minimize, in the vein of an error term when evaluating the performance of a machine learning model.

We calculate regret as the difference between the rewards we would expect an optimal strategy to collect and the actual rewards we have collected so far. In economics terms, regret is a measure of lost utility.

Utility is a term taken from economics and refers in reinforcement learning problems to the value given by the Q-function.