Neural Network Programming with TensorFlow

Defining cost and optimizer

Apply Softmax on the output from layer_fc2 (the fully connected second layer).

In mathematics, the softmax function, or normalized exponential function,^[1]:198 is a generalization of the logistic function that squashes a K-dimensional vector Z of arbitrary real values to a K-dimensional vector σ(z) of real values in the range [0, 1] that add up to 1. The function is given by the following formula:

y_pred = tf.nn.softmax(layer_fc2)y_pred_cls = tf.argmax(y_pred, dimension=1)

Calculate the cross entropy:

cross_entropy = tf.nn.softmax_cross_entropy_with_logits(  logits=layer_fc2,  labels=y_true)cost = tf.reduce_mean(cross_entropy) ...