Hands-On Q-Learning with Python

About experience replay

Why are we choosing to use experience replay? What advantage does it provide over updating the Q-values at every iteration?

Let's first take a look at the loss function for the deep Q-network:

Essentially, instead of running an update on the Q-table every time we take an action and getting a max Q-value for the next state, we store the results of each action and then run updates in randomly-chosen batches all at once.

One advantage this provides is the reduction of noise in the model. There are some updates we don't want to incorporate into our Q-function, because they are based on noise, or on a random action that ...