book

Nonlinear Parameter Optimization Using R Tools

by John C. Nash

May 2014

Intermediate to advanced

304 pages

7h 37m

English

Wiley

Read now

Unlock full access

1.1 The general optimization problem1.2 Why the general problem is generally uninteresting1.3 (Non-)Linearity1.4 Objective function properties1.5 Constraint types1.6 Solving sets of equations1.7 Conditions for optimality1.8 Other classificationsReferences
2.1 Methods that use the gradient2.2 Newton-like methods2.3 The promise of Newton's method2.4 Caution: convergence versus termination2.5 Difficulties with Newton's method2.6 Least squares: Gauss–Newton methods2.7 Quasi-Newton or variable metric method2.8 Conjugate gradient and related methods2.9 Other gradient methods2.10 Derivative-free methods2.11 Stochastic methods2.12 Constraint-based methods—mathematical programmingReferences
3.1 Perspective3.2 Issues of choice3.3 Software issues3.4 Specifying the objective and constraints to the optimizer3.5 Communicating exogenous data to problem definition functions3.6 Masked (temporarily fixed) optimization parameters3.7 Dealing with inadmissible results3.8 Providing derivatives for functions3.9 Derivative approximations when there are constraints3.10 Scaling of parameters and function3.11 Normal ending of computations3.12 Termination tests—abnormal ending3.13 Output to monitor progress of calculations3.14 Output of the optimization results3.15 Controls for the optimizer3.16 Default control settings3.17 Measuring performance3.18 The optimization interfaceReferences
4.1 Roots4.2 Equations in one variable4.3 Some examples4.4 Approaches to solving 1D root-finding problems4.5 What can go wrong?4.6 Being a smart user of root-finding programs4.7 Conclusions and extensionsReferences
5.1 The optimize() function5.2 Using a root-finder5.3 But where is the minimum?5.4 Ideas for 1D minimizers5.5 The line-search subproblemReferences

6.1 nls() from package stats6.2 A more difficult case6.3 The structure of the nls() solution6.4 Concerns with nls()6.5 Some ancillary tools for nonlinear least squares6.6 Minimizing R functions that compute sums of squares6.7 Choosing an approach6.8 Separable sums of squares problems6.9 Strategies for nonlinear least squaresReferences
7.1 Packages and methods for nonlinear equations7.2 A simple example to compare approaches7.3 A statistical exampleReferences
8.1 optim()8.2 nlm()8.3 nlminb()8.4 Using the base optimization toolsReferences
9.1 Package optimx9.2 Some other function minimization packages9.3 Should we replace optim() routines?References
10.1 Why and how10.2 Analytic derivatives—by hand10.3 Analytic derivatives—tools10.4 Examples of use of R tools for differentiation10.5 Simple numerical derivatives10.6 Improved numerical derivative approximations10.7 Strategy and tactics for derivativesReferences
11.1 Single bound: use of a logarithmic transformation11.2 Interval bounds: Use of a hyperbolic transformation11.3 Setting the objective large when bounds are violated11.4 An active set approach11.5 Checking bounds11.6 The importance of using bounds intelligently11.7 Post-solution information for bounded problemsAppendix 11.A Function transfiniteReferences
12.1 An example12.2 Specifying the objective12.3 Masks for nonlinear least squares12.4 Other approaches to masksReferences
13.1 Equality constraints13.2 Sumscale problems13.3 Inequality constraints13.4 A perspective on penalty function ideas13.5 AssessmentReferences
14.1 Statistical applications of math programming14.2 R packages for math programming14.3 Example problem: L1 regression14.4 Example problem: minimax regression14.5 Nonlinear quantile regression14.6 Polynomial approximationReferences
15.1 Panorama of methods15.2 R packages for global and stochastic optimization15.3 An example problem15.4 Multiple starting valuesReferences
16.1 Why scale or reparameterize?16.2 Formalities of scaling and reparameterization16.3 Hobbs' weed infestation example16.4 The KKT conditions and scaling16.5 Reparameterization of the weeds problem16.6 Scale change across the parameter space16.7 Robustness of methods to starting points16.8 Strategies for scalingReferences
17.1 Particular requirements17.2 Starting values for iterative methods17.3 KKT conditions17.4 Search testsReferences
18.1 Timing and profiling18.2 Profiling18.3 More speedups of R computations18.4 External language compiled functions18.5 Deciding when we are finishedReferences
19.1 Mechanisms to link R to external software19.2 Prepackaged links to external optimization tools19.3 Strategy for using external toolsReferences
20.1 The model20.2 Background20.3 The likelihood function20.4 A first try at minimization20.5 Attempts with optimx20.6 Using nonlinear least squares20.7 CommentaryReference
21.1 Maximum likelihood21.2 Generalized nonlinear models21.3 Systems of equations21.4 Additional nonlinear least squares tools21.5 Nonnegative least squares21.6 Noisy objective functions21.7 Moving forwardReferences

Content preview from Nonlinear Parameter Optimization Using R Tools

Chapter 9Add-in function minimization packages for R

CRAN (Comprehensive R Archive Network) and R-Forge have a number of packages for function minimization. Indeed, I am a developer involved in several of them. This chapter explores some of these. Truthfully, there is more material than can be sensibly included in one book, let alone one chapter of a book, but we will do our best to provide a decent overview.

9.1 Package `optimx`

The package optimx (Nash and Varadhan, 2011a, 2011b) is a wrapper package for a number of optimization routines, including several of those included in the base system. One purpose is to unify the calls to a number of useful optimizers into a common syntax and thereby allow their performance comparison in an easy way. This ability for comparison makes the package a useful introduction to a number of the packages it calls. Different syntaxes for the calls to different routines are a great nuisance for the user and a frequent source of error.

However, optimx is more than simply a tool to unify the calling sequence syntax. The design incorporates several important concepts that are discussed elsewhere in this book (see Chapter 3).

A number of initial tests of the function, gradient, and scaling are available before optimization is attempted.
After optimizers are complete, Kuhn–Karush–Tucker verification of the solutions is the default behavior if the number of parameters is not too large.
summary(), coef(), and print() methods are available for the returned ...