Practical Convolutional Neural Networks

Calculating loss 

For the discriminator, the total loss is the sum of the losses for real and fake images. The losses will be sigmoid cross-entropyies, which we can get using the TensorFlow tf.nn.sigmoid_cross_entropy_with_logits. Then we compute the mean for all the images in the batch. So the losses will look like this:

tf.reduce_mean(tf.nn.sigmoid_cross_entropy_with_logits(logits=logits, labels=labels))

To help the discriminator generalize better, the labels can be reduced a bit from 1.0 to 0.9, by for example, using the parameter smooth. This is known as label smoothing, and is typically used with classifiers to improve performance. The discriminator loss for the fake data is similar. The logits are d_logits_fake, which we got from passing ...