book

Nonlinear Parameter Optimization Using R Tools

by John C. Nash

May 2014

Intermediate to advanced

304 pages

7h 37m

English

Wiley

Read now

Unlock full access

1.1 The general optimization problem1.2 Why the general problem is generally uninteresting1.3 (Non-)Linearity1.4 Objective function properties1.5 Constraint types1.6 Solving sets of equations1.7 Conditions for optimality1.8 Other classificationsReferences
2.1 Methods that use the gradient2.2 Newton-like methods2.3 The promise of Newton's method2.4 Caution: convergence versus termination2.5 Difficulties with Newton's method2.6 Least squares: Gauss–Newton methods2.7 Quasi-Newton or variable metric method2.8 Conjugate gradient and related methods2.9 Other gradient methods2.10 Derivative-free methods2.11 Stochastic methods2.12 Constraint-based methods—mathematical programmingReferences
3.1 Perspective3.2 Issues of choice3.3 Software issues3.4 Specifying the objective and constraints to the optimizer3.5 Communicating exogenous data to problem definition functions3.6 Masked (temporarily fixed) optimization parameters3.7 Dealing with inadmissible results3.8 Providing derivatives for functions3.9 Derivative approximations when there are constraints3.10 Scaling of parameters and function3.11 Normal ending of computations3.12 Termination tests—abnormal ending3.13 Output to monitor progress of calculations3.14 Output of the optimization results3.15 Controls for the optimizer3.16 Default control settings3.17 Measuring performance3.18 The optimization interfaceReferences
4.1 Roots4.2 Equations in one variable4.3 Some examples4.4 Approaches to solving 1D root-finding problems4.5 What can go wrong?4.6 Being a smart user of root-finding programs4.7 Conclusions and extensionsReferences
5.1 The optimize() function5.2 Using a root-finder5.3 But where is the minimum?5.4 Ideas for 1D minimizers5.5 The line-search subproblemReferences

6.1 nls() from package stats6.2 A more difficult case6.3 The structure of the nls() solution6.4 Concerns with nls()6.5 Some ancillary tools for nonlinear least squares6.6 Minimizing R functions that compute sums of squares6.7 Choosing an approach6.8 Separable sums of squares problems6.9 Strategies for nonlinear least squaresReferences
7.1 Packages and methods for nonlinear equations7.2 A simple example to compare approaches7.3 A statistical exampleReferences
8.1 optim()8.2 nlm()8.3 nlminb()8.4 Using the base optimization toolsReferences
9.1 Package optimx9.2 Some other function minimization packages9.3 Should we replace optim() routines?References
10.1 Why and how10.2 Analytic derivatives—by hand10.3 Analytic derivatives—tools10.4 Examples of use of R tools for differentiation10.5 Simple numerical derivatives10.6 Improved numerical derivative approximations10.7 Strategy and tactics for derivativesReferences
11.1 Single bound: use of a logarithmic transformation11.2 Interval bounds: Use of a hyperbolic transformation11.3 Setting the objective large when bounds are violated11.4 An active set approach11.5 Checking bounds11.6 The importance of using bounds intelligently11.7 Post-solution information for bounded problemsAppendix 11.A Function transfiniteReferences
12.1 An example12.2 Specifying the objective12.3 Masks for nonlinear least squares12.4 Other approaches to masksReferences
13.1 Equality constraints13.2 Sumscale problems13.3 Inequality constraints13.4 A perspective on penalty function ideas13.5 AssessmentReferences
14.1 Statistical applications of math programming14.2 R packages for math programming14.3 Example problem: L1 regression14.4 Example problem: minimax regression14.5 Nonlinear quantile regression14.6 Polynomial approximationReferences
15.1 Panorama of methods15.2 R packages for global and stochastic optimization15.3 An example problem15.4 Multiple starting valuesReferences
16.1 Why scale or reparameterize?16.2 Formalities of scaling and reparameterization16.3 Hobbs' weed infestation example16.4 The KKT conditions and scaling16.5 Reparameterization of the weeds problem16.6 Scale change across the parameter space16.7 Robustness of methods to starting points16.8 Strategies for scalingReferences
17.1 Particular requirements17.2 Starting values for iterative methods17.3 KKT conditions17.4 Search testsReferences
18.1 Timing and profiling18.2 Profiling18.3 More speedups of R computations18.4 External language compiled functions18.5 Deciding when we are finishedReferences
19.1 Mechanisms to link R to external software19.2 Prepackaged links to external optimization tools19.3 Strategy for using external toolsReferences
20.1 The model20.2 Background20.3 The likelihood function20.4 A first try at minimization20.5 Attempts with optimx20.6 Using nonlinear least squares20.7 CommentaryReference
21.1 Maximum likelihood21.2 Generalized nonlinear models21.3 Systems of equations21.4 Additional nonlinear least squares tools21.5 Nonnegative least squares21.6 Noisy objective functions21.7 Moving forwardReferences

Overview

Nonlinear Parameter Optimization Using R

John C. Nash, Telfer School of Management, University of Ottawa, Canada

A systematic and comprehensive treatment of optimization software using R

In recent decades, optimization techniques have been streamlined by computational and artificial intelligence methods to analyze more variables, especially under non-linear, multivariable conditions, more quickly than ever before.

Optimization is an important tool for decision science and for the analysis of physical systems used in engineering. Nonlinear Parameter Optimization with R explores the principal tools available in R for function minimization, optimization, and nonlinear parameter determination and features numerous examples throughout.

Nonlinear Parameter Optimization with R:

Provides a comprehensive treatment of optimization techniques

Examines optimization problems that arise in statistics and how to solve them using R

Enables researchers and practitioners to solve parameter determination problems

Presents traditional methods as well as recent developments in R

Is supported by an accompanying website featuring R code, examples and datasets

Researchers and practitioners who have to solve parameter determination problems who are users of R but are novices in the field optimization or function minimization will benefit from this book. It will also be useful for scientists building and estimating nonlinear models in various fields such as hydrology, sports forecasting, ecology, chemical engineering, pharmaco-kinetics, agriculture, economics and statistics.