Engineering Optimization

18Enhancements to Optimizers

18.1 Introduction

The objective is to improve the optimizers, to optimize the optimization algorithms. Specific metrics would be to maximize the probability of finding the global optimum and robustness to aberrations (constraints, nonlinearities, stochastic response, discontinuities, flat spots, etc.) while minimizing computational work (of both the algorithm and number of function evaluations) and minimizing complexity (for either programmer or user). In any one optimizer, there seem to be dozens of enhancements the lead to improvements.

This chapter summarizes a few techniques that I think are relevant, archetypical, and broadly functional.

18.2 Criteria for Replicate Trials

Consider a deterministic (not stochastic) response. One run of an optimizer may converge at an optimum, but it may be a local optimum, not the global. Another run from a different initialization may lead to the same local optimum. Or it might be the single global optimum has been found twice. How can one tell whether the global has been found? That is one question.

Even if there is only one optimum, the global, successive runs from independent initializations will each converge in the vicinity of the optimum, but not exactly on the true DV* spot. Consequently, each solution will likely have slightly different DV* and OF* values. How can one tell whether the different values indicate a common solution or different solutions? That is another question.

The true optimum will ...