Hands-On Neural Network Programming with C#

Choosing an optimizer

When faced with a new optimization problem, the first optimizer you may want to try is the PS method, which is often sufficient and has the advantage of converging (or stagnating) very quickly. In addition, PS does not have any behavioral parameters that need tuning, so it either works or it doesn't. If the PS method fails at optimizing your problem, you may want to try the LUS method. You may need to run both PS and LUS several times as they may converge to sub-optimal solutions. If PS and LUS both fail, you may try the DE, MOL, or PSO methods and experiment with their behavioral parameters.

As a rule of thumb, the PS and LUS methods stagnate rather quickly, say, after  iterations, where  is the dimensionality of the ...