Hands-On Q-Learning with Python

Fixed Q-targets

The fixed Q-targets solution was introduced by the DeepMind team. Using two DQNs instead of one, this method keeps the target values of one network (called the target network) fixed and periodically updates the network weights.

Here's a sample implementation of a target network in Keras:

class DQN:    def __init__(self, state_size, action_size):         self.model = self._build_model()         self.target_model = self._build_model()         self.update_target_model()        def update_target_model(self):        self.target_model.set_weights(self.model.get_weights())

For most of the training process, the weights in the target network are fixed, taking away the problem of trying to predict a moving target.