Gradient descent is a method that's used to calculate the best direction to move in when we're searching for the solution to a minimization/maximization problem. This method suggests the direction to follow when we're updating the model parameters: the direction that's found, depending on the input data that's used, is the direction of the steepest descent of the loss surface. The data that's used is of extreme importance since it follows the evaluation of the loss function and therefore the surface that's used to evaluate the update direction.
The update direction is given by the gradient of the loss function. It's known from calculus that the derivative operation for a single variable differentiable function, , in point ...
