Hands-On Intelligent Agents with OpenAI Gym

Dueling networks

Dueling networks is a neural network architecture designed for value-based reinforcement learning. The name dueling stems from the main feature of this architecture, which is that there are two streams of computations, one for the value function and the other for the advantage. The following diagram, from a research paper (https://arxiv.org/pdf/1511.06581.pdf), shows the comparison of the dueling network architecture (the network shown at the bottom of the diagram) with the typical DQN architecture (shown at the top of the diagram):

The convolutional layers that encode features are shared by both the value and advantage ...