10.1 Introduction

The class of conjugate direction methods can be viewed as being intermediate between the method of steepest descent and Newton’s method. The conjugate direction methods have the following properties:

1. Solve quadratics of n variables in n steps.
2. The usual implementation, the conjugate gradient algorithm, requires no Hessian matrix evaluations.
3. No matrix inversion and no storage of an n × n matrix are required.

The conjugate direction methods typically perform better than the method of steepest descent, but not as well as Newton’s method. As we saw from the method of steepest descent and Newton’s method, the crucial factor in the efficiency of an iterative search method is the direction of search at each iteration. For a quadratic function of n variables f(x) = xQxxb, x n, Q = Q > 0, the best direction of search, as we shall ...

Get An Introduction to Optimization, 4th Edition now with O’Reilly online learning.

O’Reilly members experience live online training, plus books, videos, and digital content from 200+ publishers.