Hands-On Q-Learning with Python

Dueling deep Q-networks

The Q-value of a state can be decomposed to two values:

• V(s): The value of currently being in state s
• A(s, a): The advantage of taking action a from state s over all the other actions the agent could take from state s

Dueling deep Q-networks separate out the estimators of these two values, using a different DQN to calculate each of the values for V(s) and A(s, a). They then combine the two values into an overall Q-value using an aggregating layer.

Here's a sample network architecture to accomplish this:

One positive ...