Applied Numerical Methods Using MATLAB, 2nd Edition

7Optimization

Chapter Outline

• 7.1 Unconstrained Optimization
  • 7.1.1 Golden Search Method
  • 7.1.2 Quadratic Approximation Method
  • 7.1.3 Nelder‐Mead Method
  • 7.1.4 Steepest Descent Method
  • 7.1.5 Newton Method
  • 7.1.6 Conjugate Gradient Method
  • 7.1.7 Simulated Annealing
  • 7.1.8 Genetic Algorithm
• 7.2 Constrained Optimization
  • 7.2.1 Lagrange Multiplier Method
  • 7.2.2 Penalty Function Method
• 7.3 MATLAB Built‐In Functions for Optimization
  • 7.3.1 Unconstrained Optimization
  • 7.3.2 Constrained Optimization
  • 7.3.3 Linear Programming (LP)
  • 7.3.4 Mixed Integer Linear Programming (MILP)
• 7.4 Neural Network^[K‐1]
• 7.5 Adaptive Filter^[Y‐3]
• 7.6 Recursive Least Square Estimation (RLSE)^[Y‐3]
• Problems

Optimization involves finding the minimum/maximum of an objective (or cost) function f(x) subject to some constraint x ∈ S. If there is no constraint for x to satisfy, or equivalently, S is the universe, it is called an unconstrained optimization and otherwise, it is a constrained optimization. In this chapter, we will cover several unconstrained optimization techniques such as the golden search method, the quadratic approximation method, Nelder‐Mead method, the steepest descent method, Newton method, simulated‐annealing (SA) method, and genetic algorithm (GA). As for constrained optimization, we will only introduce the MATLAB built‐in functions together with the functions for unconstrained optimization. Note that we do not have to distinguish maximization and minimization because maximizing f(x) is equivalent ...