Hands-On Convolutional Neural Networks with TensorFlow

Loss interpretability

One of the problems while training GANs is that the values of both the generator and discriminator loss do not have a noticeable effect. It's not like training a classifier that you just wait for the loss to drop to see whether your model is training.

With GANs, the loss values' dropping does not necessarily mean that the model is training:

From many people's experiments and research, here are some tips on how to use the GAN loss values:

• You don't want that the discriminator loss to go down really fast, because it will not be able to provide feedback to the generator to improve it.
• If the generator loss falls quickly, ...