Hands-On Deep Learning Algorithms with Python

Computing loss and backpropagation

Next, we'll define our loss function. We'll use softmax cross-entropy as our loss function. TensorFlow provides the tf.nn.softmax_cross_entropy_with_logits() function for computing softmax cross-entropy loss. It takes two parameters as inputs, logits and labels:

• The logits parameter specifies the logits predicted by our network; for example, y_hat
• The labels parameter specifies the actual labels; for example, true labels, Y

We take the mean of the loss function using tf.reduce_mean():

with tf.name_scope('Loss'):        loss = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits=y_hat,labels=Y))

Now, we need to minimize the loss using backpropagation. Don't worry! We don't have to calculate the derivatives ...