Deep Learning Quick Reference

Stability

Training a GAN requires a careful balancing act between the discriminator and generator. The discriminator and generator are both fighting against each other for deep network supremacy. On the other hand, they also need each other to learn and grow. In order for this to work, neither can overpower the other one.

In an unstable GAN, the discriminator might overpower the generator, and become absolutely certain that the generator is fake. The loss goes to zero, and there is no gradient available to be sent to the generator, so it can no longer improve. Game over for the network. The best way to address this is to lower the learning rate of the discriminator. You might also try reducing the number of neurons in the overall discriminator ...