Reinforcement Learning with TensorFlow

Double DQN

The reason behind the use of Double DQN (DDQN) is that the regular DQN overestimates the Q-values of potential actions to take in a given state. Overestimation is not equal across all the actions in a regular DQN. Therefore the issue persisted: otherwise, equal estimation across all actions would not have been an issue. As a result, certain suboptimal actions were getting higher values so the time to learn optimal policy increased. This led to small modifications in our regular DQN architecture and it resulted in what we call DDQN , that is, double deep Q-network.

In DDQN, instead of taking the max over Q-values while computing the target Q-value during training, we use a primary network to choose the action and target network ...