Deep Learning Essentials

Dueling DQN

So far we have seen that most Q-learning focuses on learning a state-action table Q (s,a), which measures how good a particular action a is in any given state s. This is achieved by learning this function jointly to optimize both being present in a given state and taking a particular action from that state. From a learning perspective, it might be easier to simply learn the usefulness of being in a given state without caring much about the action values. Decoupling state utility from action value might help model these functions robustly. This forms the core idea behind a dueling DQN architecture.

Dueling DQN decomposes the Q-learning function into two separate functions: (a) Value function: V(s) and (b) Advantage function: