Python Natural Language Processing

Backpropagation

We have already seen how errors have been calculated using the L2 loss function, and the L2 loss function wants to minimize the squared differences between the estimated and existing target values. We will apply the same concepts to the multi-layer neural network. So, we need to define the loss function as well as we to take the gradient of the function and update the weight of the neural network in order to generate a minimum error value. Here, our input and output is vectors.

Refer to Figure 6.27 to see the neural network structure, and Figure 6.28 shows what equations we need to apply to calculate error functions in a multi-layer neural network:

Figure 6.27: Multi-layer neural network for calculating error functions