Hands-On Q-Learning with Python

Decaying epsilon

The more familiar your agent becomes with its environment, the less exploration we want it to do. As it discovers more and more rewards, the odds that it will discover actions with higher reward values than the ones it has already discovered begin to decrease. It should start to increasingly stick with actions it knows are highly-valued and do less and less exploration.

This concept is called exploration versus exploitation. Exploration refers to discovering new states that may be higher-valued than the ones our agent has already seen, and exploitation means visiting the highest-valued states it has seen to benefit from the rewards it already knows it will collect there.

One popular illustration of this problem is the multi-armed ...