Optimization

16 Optimization: Algorithms and Applications

Maximize

f(x)

is the same as

Minimize

−f(x)

1.6 Gradient Vector, Directional Derivative,

and Hessian Matrix

The derivative or gradient of a function f(x) at a point x, generally denoted by

f′(x), is the slope of the tangent (see Figure 1.12) at that point. That is,

f′(x) = tan θ (1.35)

where θ is the angle measured by the tangent with respect to the horizon-

tal. Along the gradient direction, there is the maximum change in the value

of the function. Thus, gradient information provides the necessary search

direction to locate the maximum or minimum of the function.

−2 −1.5 −1 −0.5 0 0.5 1 1.5 2

−8

−6