Python Natural Language Processing

Softmax function

In this section, we will look at the softmax function. The softmax function is used for converting output vectors into a probability form. We are using the softmax function because we want to convert out last layer output in terms of probability and softmax function can easily convert vector values into probability values. Here, the output of the k^th neuron will be computed by the following equation, where activation(n) represents the activation value of the nth output layer neuron:

By using this equation, we can calculate the probabilities for eight words in the corpus and the probability values are given as follows:

[ 0.143073 ...