Hands-On Intelligent Agents with OpenAI Gym

Prioritized experience replay

When we implemented deep Q-learning in Chapter 6, Implementing an Intelligent Agent for Optimal Discrete Control Using Deep Q-Learning, we used an experience replay memory to store and retrieve sampled transition experience. In our implementation, and in the DQN algorithm, the experiences from the replay memory buffer are sampled uniformly. Intuitively, we would want to sample those experiences more frequently, as there is much to learn. Prioritized experience replay samples transition with probability relative to the last encountered absolute TD error, given by the following equation:

Here, is a hyperparameter ...