Hands-On Neural Network Programming with C#

Meta-optimization

Optimization methods usually have several user-defined parameters that govern the behavior and efficacy of the optimization method. These are called the optimizer's behavioral or control parameters. Finding a good choice of these behavioral parameters has previously been done manually by hand-tuning, and sometimes even by using coarse mathematical analysis. It has also become a common belief among researchers that behavioral parameters can be adapted during optimization to improve overall optimization performance; however, this has been demonstrated to be mostly unlikely. Tuning behavioral parameters can be considered an optimization problem and hence can be solved by an overlaid optimization method. This is known here as ...