Hands-On Q-Learning with Python

Decaying epsilon

As we learn more and more about our environment and find out where the high-valued actions are, we want to do more exploitation of those high-valued actions and less exploration of other potential actions that we may not have discovered yet. There are several ways to decay epsilon and improve the performance of our model.

One option we can use is to decay epsilon in a straight line, independent of the values in the Q-table. That's the option we'll be using at first:

Q = np.zeros([env.observation_space.n, env.action_space.n])gamma = 0.1alpha = 0.1epsilon = 0.1epsilon_decay = 0.99 #decay factor total_epochs = 0episodes = 10000for episode in range(episodes):    epochs = 0    reward = 0 epsilon = epsilon * epsilon_decay #decay step ...