Reinforcement Learning Algorithms with Python

Results

To see if DQN actually overestimates the Q-values in respect to DDQN, we reported the Q-value plot in the following diagram. We also included the results of DQN (the orange line) so that we have a direct comparison between the two algorithms:

Figure 5.8. A plot of the estimated training Q-values. The DDQN values are plotted in blue and the DQN values are plotted in orange. The x axis represents the number of steps

The performance of both DDQN (the blue line) and DQN (the orange line), which are represented by the average reward of the test games, is as follows:

For all the color references mentioned in the chapter, please refer to ...