Multivariate Optimization

Building upon the conceptual framework established in the previous chapter, the present chapter introduces the reader to the basic methods of nonlinear optimization, in particular the steepest descent and Newton’s methods. We also witness the development of a mixed or hybrid method, and its application in the ubiquitous nonlinear least squares and equation solving problems.

Direct Methods

Direct search methods or derivative-free methods are important for two reasons. First, they are simple to understand and implement. Besides, if the objective function is not differentiable, then these methods also have an advantage, when derivative-based methods are likely to misrepresent the function profile.1

Cyclic coordinate ...

Get Applied Mathematical Methods now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.