Hands-On Machine Learning on Google Cloud Platform

Softmax activation

From the preceding section, we should notice that in the case of categorical variable prediction, the number of units in the output layer would be the same as the number of distinct values in the dependent variable.

Also, note that the predicted value cannot be greater than 1 or less than 0 for any of the units in the output layer. At the same time, the sum of the values across all nodes in the output should be equal to 1.

For example, let's say the output across two nodes of output is -1 and 5. Given that the expected value of outputs should be between 0 and 1 (the probability of an event happening), we pass the output values through softmax activation, as follows:

• Pass the values through an exponential function:

exp(-1) ...