Optimization
We have to figure out a way to minimize the total loss function and it can be achieved by changing the weight. It can be done by using a crude method like modifying the parameter W over a range of -500 to 500 with a step 0.001. It will help us to find a point where the sum of squares of error becomes 0 or minimum.
But this approach will work out in this scenario because we don't have too many parameters here and computation won't be too challenging. However, when we have a number of parameters, the computation would take a hit.
Here, mathematics comes to our rescue in the form of differentiation (maxima and minima approach) in order to optimize the weights. The derivative of a function at a certain point gives the rate at which ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access