TensorFlow 2.0 Quick Start Guide

Softmax layer

A softmax layer is a layer where the activation of each output unit corresponds to the probability that the output unit matches a given label. The output neuron with the highest activation value is, therefore, the prediction of the net. It is used when the classes being learned are mutually exclusive, so that the probabilities output by the softmax layer total 1.

It is implemented as an activation on a dense layer.

Hence, for example, we have the following:

model2.add(tf.keras.layers.Dense(10,activation=tf.nn.softmax))

This would add a dense softmax layer with 10 neurons where the activations of the neurons would total 1.

Next, we will talk a little more about activation functions.