40 Algorithms Every Programmer Should Know

Softmax

Sometimes we need more than two levels for the output of the activation function. Softmax is an activation function that provides us with more than two levels for the output. It is best suited to multiclass classification problems. Let's assume that we have n classes. We have input values. The input values map the classes as follows:

x = {x⁽¹⁾,x⁽²⁾,....x⁽ⁿ⁾}

Softmax operates on probability theory. The output probability of the e^th class of the softmax is calculated as follows:

For binary classifiers, the activation function in the final layer will be sigmoid, and for multiclass classifiers it will be softmax.