Applied Mathematical Methods

22

Multivariate Optimization

Building upon the conceptual framework established in the previous chapter, the present chapter introduces the reader to the basic methods of nonlinear optimization, in particular the steepest descent and Newton’s methods. We also witness the development of a mixed or hybrid method, and its application in the ubiquitous nonlinear least squares and equation solving problems.

Direct Methods

Direct search methods or derivative-free methods are important for two reasons. First, they are simple to understand and implement. Besides, if the objective function is not differentiable, then these methods also have an advantage, when derivative-based methods are likely to misrepresent the function profile.¹

Cyclic coordinate ...