Python Natural Language Processing

Understanding gradient descent in word2vec

Now, let's understand what we are going to do with the L2 function and how it will be useful in achieving an accurate output.

As we said earlier, we want to minimize this function value so, we can accurately predict the target value, and to achieve this we will take partial derivative of the L2 function equation given in Figure 6.21 with respect to y. The procedure of deriving derivation and then by using this derivation trying to minimize error values, is called gradient descent.

In that case, the result is given in Figure 6.22:

Figure 6.22: Result of Partial derivatives of L2 loss function with ...