An Introduction to Optimization, 4th Edition

CHAPTER 10

CONJUGATE DIRECTION METHODS

10.1 Introduction

The class of conjugate direction methods can be viewed as being intermediate between the method of steepest descent and Newton’s method. The conjugate direction methods have the following properties:

1. Solve quadratics of n variables in n steps.

2. The usual implementation, the conjugate gradient algorithm, requires no Hessian matrix evaluations.

3. No matrix inversion and no storage of an n × n matrix are required.

The conjugate direction methods typically perform better than the method of steepest descent, but not as well as Newton’s method. As we saw from the method of steepest descent and Newton’s method, the crucial factor in the efficiency of an iterative search method is the direction of search at each iteration. For a quadratic function of n variables f(x) = xQx − xb, x ⁿ, Q = Q > 0, the best direction of search, as we shall ...