Having presented some of the problems that fall under the umbrella that is the title of this book, we now turn to consider different ways to structure software to try to find solutions to those problems. For discussion purposes and because it is the class of problems with which I have the most experience, we will primarily focus on function minimization with at most quite simple constraints. In the statistical community, many people call this optimization, while those from engineering and economics would expect problems with many more constraints but a possibly simpler objective function. As R is largely aimed at statistical computations, the treatment here will lean in that direction.
In a fairly long career in this area, I have found that there is a common love of “speed.” More correctly, people do not want to feel that they are waiting for their calculations to finish. If they do find that they are waiting, they may begin to judge methods on the basis of timings, sometimes surprisingly few and poorly conducted timings.
While sharing a strong interest in computational efficiency, I take the following views.