Hands-On Generative Adversarial Networks with Keras

Vanishing gradients

In their paper Towards Principled Methods for Training Generative Adversarial Networks, Martin Ajovsky and Léon Bottou shed theoretical light onto vanishing gradients and unstable behavior associated with the original GAN and its loss. Here, we briefly summarize a few important conclusions from the paper. Let's start by remembering that the discriminator is trained to maximize the following:

When trained until convergence, the loss of the discriminator will be minimized be when for real samples and for fake samples . This ...