book

Nonlinear Parameter Optimization Using R Tools

by John C. Nash

May 2014

Intermediate to advanced

304 pages

7h 37m

English

Wiley

Read now

Unlock full access

1.1 The general optimization problem1.2 Why the general problem is generally uninteresting1.3 (Non-)Linearity1.4 Objective function properties1.5 Constraint types1.6 Solving sets of equations1.7 Conditions for optimality1.8 Other classificationsReferences
2.1 Methods that use the gradient2.2 Newton-like methods2.3 The promise of Newton's method2.4 Caution: convergence versus termination2.5 Difficulties with Newton's method2.6 Least squares: Gauss–Newton methods2.7 Quasi-Newton or variable metric method2.8 Conjugate gradient and related methods2.9 Other gradient methods2.10 Derivative-free methods2.11 Stochastic methods2.12 Constraint-based methods—mathematical programmingReferences
3.1 Perspective3.2 Issues of choice3.3 Software issues3.4 Specifying the objective and constraints to the optimizer3.5 Communicating exogenous data to problem definition functions3.6 Masked (temporarily fixed) optimization parameters3.7 Dealing with inadmissible results3.8 Providing derivatives for functions3.9 Derivative approximations when there are constraints3.10 Scaling of parameters and function3.11 Normal ending of computations3.12 Termination tests—abnormal ending3.13 Output to monitor progress of calculations3.14 Output of the optimization results3.15 Controls for the optimizer3.16 Default control settings3.17 Measuring performance3.18 The optimization interfaceReferences
4.1 Roots4.2 Equations in one variable4.3 Some examples4.4 Approaches to solving 1D root-finding problems4.5 What can go wrong?4.6 Being a smart user of root-finding programs4.7 Conclusions and extensionsReferences
5.1 The optimize() function5.2 Using a root-finder5.3 But where is the minimum?5.4 Ideas for 1D minimizers5.5 The line-search subproblemReferences

6.1 nls() from package stats6.2 A more difficult case6.3 The structure of the nls() solution6.4 Concerns with nls()6.5 Some ancillary tools for nonlinear least squares6.6 Minimizing R functions that compute sums of squares6.7 Choosing an approach6.8 Separable sums of squares problems6.9 Strategies for nonlinear least squaresReferences
7.1 Packages and methods for nonlinear equations7.2 A simple example to compare approaches7.3 A statistical exampleReferences
8.1 optim()8.2 nlm()8.3 nlminb()8.4 Using the base optimization toolsReferences
9.1 Package optimx9.2 Some other function minimization packages9.3 Should we replace optim() routines?References
10.1 Why and how10.2 Analytic derivatives—by hand10.3 Analytic derivatives—tools10.4 Examples of use of R tools for differentiation10.5 Simple numerical derivatives10.6 Improved numerical derivative approximations10.7 Strategy and tactics for derivativesReferences
11.1 Single bound: use of a logarithmic transformation11.2 Interval bounds: Use of a hyperbolic transformation11.3 Setting the objective large when bounds are violated11.4 An active set approach11.5 Checking bounds11.6 The importance of using bounds intelligently11.7 Post-solution information for bounded problemsAppendix 11.A Function transfiniteReferences
12.1 An example12.2 Specifying the objective12.3 Masks for nonlinear least squares12.4 Other approaches to masksReferences
13.1 Equality constraints13.2 Sumscale problems13.3 Inequality constraints13.4 A perspective on penalty function ideas13.5 AssessmentReferences
14.1 Statistical applications of math programming14.2 R packages for math programming14.3 Example problem: L1 regression14.4 Example problem: minimax regression14.5 Nonlinear quantile regression14.6 Polynomial approximationReferences
15.1 Panorama of methods15.2 R packages for global and stochastic optimization15.3 An example problem15.4 Multiple starting valuesReferences
16.1 Why scale or reparameterize?16.2 Formalities of scaling and reparameterization16.3 Hobbs' weed infestation example16.4 The KKT conditions and scaling16.5 Reparameterization of the weeds problem16.6 Scale change across the parameter space16.7 Robustness of methods to starting points16.8 Strategies for scalingReferences
17.1 Particular requirements17.2 Starting values for iterative methods17.3 KKT conditions17.4 Search testsReferences
18.1 Timing and profiling18.2 Profiling18.3 More speedups of R computations18.4 External language compiled functions18.5 Deciding when we are finishedReferences
19.1 Mechanisms to link R to external software19.2 Prepackaged links to external optimization tools19.3 Strategy for using external toolsReferences
20.1 The model20.2 Background20.3 The likelihood function20.4 A first try at minimization20.5 Attempts with optimx20.6 Using nonlinear least squares20.7 CommentaryReference
21.1 Maximum likelihood21.2 Generalized nonlinear models21.3 Systems of equations21.4 Additional nonlinear least squares tools21.5 Nonnegative least squares21.6 Noisy objective functions21.7 Moving forwardReferences

Content preview from Nonlinear Parameter Optimization Using R Tools

Appendix A

R packages used in examples

The following R packages were used in the examples in this book. Sources for those not on CRAN (Comprehensive R Archive Network) are noted.

adagio: Discrete and global optimization routines
alabama: Constrained nonlinear optimization
BB: Solving and optimizing large-scale nonlinear systems
bbmle: Tools for general maximum likelihood estimation
compiler: This is now provided as a standard package in the base distribution.
DEoptim: Global optimization by differential evolution
deSolve: General solvers for initial value problems of ordinary differential equations (ODE), partial differential equations (PDE), differential algebraic equations (DAE), and delay differential equations (DDE)
expm: Matrix exponential
GA: Genetic algorithms
gaoptim: Genetic algorithm optimization for real-based and permutation-based problems
GenSA: R functions for generalized Simulated Annealing
gnm: Generalized nonlinear models
grofit: The package was developed to fit many growth curves obtained under different conditions
linprog: Linear programming/optimization
MASS: Functions and data sets to support Venables and Ripley, ‘Modern Applied Statistics with S’ (4th edition, 2002). This recommended package generally is installed with the base distribution.
maxLik: Maximum likelihood estimation
microbenchmark: Submicrosecond accurate timing functions
minpack.lm: R interface to the Levenberg–Marquardt nonlinear least squares algorithm found in MINPACK, plus support for ...