734 Chapter 16 Numerical Methods
The problem with this method is that it can be very slow. The pathological case
is the minimization of a paraboloid f(x, y) = (x/a)
2
+ y
2
,wherea is a very large
number. The level sets are ellipses that are very elongated in the x-direction. For
points not on the x-axis, the negative of the gradient vector tends to be nearly parallel
to the y-axis. The search path will zig-zag back and forth across the x-axis, taking its
time getting to the origin, where the global minimum occurs. A better approach is not
to use the gradient vector, but to use the conjugate direction. For the par aboloid, no
matter where the initial guess is, only two iterations using conjugate directions will
always end up at the origin. These directions ...