An Introduction to Optimization, 4th Edition

CHAPTER 8

GRADIENT METHODS

8.1 Introduction

In this chapter we consider a class of search methods for real-valued functions on ⁿ. These methods use the gradient of the given function. In our discussion we use such terms as level sets, normal vectors, and tangent vectors. These notions were discussed in some detail in Part I.

Recall that a level set of a function f : ⁿ → is the set of points x satisfying f(x) = c for some constant c. Thus, a point x₀ ⁿ is on the level set corresponding to level c if f(x₀) = c. In the case of functions of two real variables, f : ² → , the notion of the level set is illustrated in Figure 8.1.

Figure 8.1 Constructing a level set corresponding to level c for f.

The gradient of f at x₀, denoted ∇f(x₀), if it is not a zero vector, is orthogonal to the tangent ...