Deep Learning with Keras

Balancing exploration with exploitation

Deep reinforcement learning is an example of online learning, where the training and prediction steps are interspersed. Unlike batch learning techniques where the best predictor is generated by learning on the entire training data, a predictor trained with online learning is continuously improving as it trains on new data.

Thus in the initial epochs of training, a deep Q-network gives random predictions which can give rise to poor Q-learning performance. To alleviate this, we can use a simple exploration method such as ε-greedy. In case of ε-greedy exploration, the agent chooses the action suggested by the network with probability 1-ε or an action uniformly at random otherwise. That is ...