If the machine learning algorithm is a car, then the optimization algorithm is its engine. For more information, refer to http://www.pyimagesearch.com/2016/10/10/gradient-descent-with-python/.

From your math classes, you should remember that geometrical interpretation is a derivative of a function *f(x)* is a slope at any given point (*x)* of the function. Now, if we have a function of two parameters *f(x, y)*, we can't calculate just derivative as we did previously. Nevertheless, we can calculate partial derivatives: . The vector composed of these partial derivatives is known as a gradient, and the ...