book

Nonlinear Parameter Optimization Using R Tools

by John C. Nash

May 2014

Intermediate to advanced

304 pages

7h 37m

English

Wiley

Read now

Unlock full access

1.1 The general optimization problem1.2 Why the general problem is generally uninteresting1.3 (Non-)Linearity1.4 Objective function properties1.5 Constraint types1.6 Solving sets of equations1.7 Conditions for optimality1.8 Other classificationsReferences
2.1 Methods that use the gradient2.2 Newton-like methods2.3 The promise of Newton's method2.4 Caution: convergence versus termination2.5 Difficulties with Newton's method2.6 Least squares: Gauss–Newton methods2.7 Quasi-Newton or variable metric method2.8 Conjugate gradient and related methods2.9 Other gradient methods2.10 Derivative-free methods2.11 Stochastic methods2.12 Constraint-based methods—mathematical programmingReferences
3.1 Perspective3.2 Issues of choice3.3 Software issues3.4 Specifying the objective and constraints to the optimizer3.5 Communicating exogenous data to problem definition functions3.6 Masked (temporarily fixed) optimization parameters3.7 Dealing with inadmissible results3.8 Providing derivatives for functions3.9 Derivative approximations when there are constraints3.10 Scaling of parameters and function3.11 Normal ending of computations3.12 Termination tests—abnormal ending3.13 Output to monitor progress of calculations3.14 Output of the optimization results3.15 Controls for the optimizer3.16 Default control settings3.17 Measuring performance3.18 The optimization interfaceReferences
4.1 Roots4.2 Equations in one variable4.3 Some examples4.4 Approaches to solving 1D root-finding problems4.5 What can go wrong?4.6 Being a smart user of root-finding programs4.7 Conclusions and extensionsReferences
5.1 The optimize() function5.2 Using a root-finder5.3 But where is the minimum?5.4 Ideas for 1D minimizers5.5 The line-search subproblemReferences

6.1 nls() from package stats6.2 A more difficult case6.3 The structure of the nls() solution6.4 Concerns with nls()6.5 Some ancillary tools for nonlinear least squares6.6 Minimizing R functions that compute sums of squares6.7 Choosing an approach6.8 Separable sums of squares problems6.9 Strategies for nonlinear least squaresReferences
7.1 Packages and methods for nonlinear equations7.2 A simple example to compare approaches7.3 A statistical exampleReferences
8.1 optim()8.2 nlm()8.3 nlminb()8.4 Using the base optimization toolsReferences
9.1 Package optimx9.2 Some other function minimization packages9.3 Should we replace optim() routines?References
10.1 Why and how10.2 Analytic derivatives—by hand10.3 Analytic derivatives—tools10.4 Examples of use of R tools for differentiation10.5 Simple numerical derivatives10.6 Improved numerical derivative approximations10.7 Strategy and tactics for derivativesReferences
11.1 Single bound: use of a logarithmic transformation11.2 Interval bounds: Use of a hyperbolic transformation11.3 Setting the objective large when bounds are violated11.4 An active set approach11.5 Checking bounds11.6 The importance of using bounds intelligently11.7 Post-solution information for bounded problemsAppendix 11.A Function transfiniteReferences
12.1 An example12.2 Specifying the objective12.3 Masks for nonlinear least squares12.4 Other approaches to masksReferences
13.1 Equality constraints13.2 Sumscale problems13.3 Inequality constraints13.4 A perspective on penalty function ideas13.5 AssessmentReferences
14.1 Statistical applications of math programming14.2 R packages for math programming14.3 Example problem: L1 regression14.4 Example problem: minimax regression14.5 Nonlinear quantile regression14.6 Polynomial approximationReferences
15.1 Panorama of methods15.2 R packages for global and stochastic optimization15.3 An example problem15.4 Multiple starting valuesReferences
16.1 Why scale or reparameterize?16.2 Formalities of scaling and reparameterization16.3 Hobbs' weed infestation example16.4 The KKT conditions and scaling16.5 Reparameterization of the weeds problem16.6 Scale change across the parameter space16.7 Robustness of methods to starting points16.8 Strategies for scalingReferences
17.1 Particular requirements17.2 Starting values for iterative methods17.3 KKT conditions17.4 Search testsReferences
18.1 Timing and profiling18.2 Profiling18.3 More speedups of R computations18.4 External language compiled functions18.5 Deciding when we are finishedReferences
19.1 Mechanisms to link R to external software19.2 Prepackaged links to external optimization tools19.3 Strategy for using external toolsReferences
20.1 The model20.2 Background20.3 The likelihood function20.4 A first try at minimization20.5 Attempts with optimx20.6 Using nonlinear least squares20.7 CommentaryReference
21.1 Maximum likelihood21.2 Generalized nonlinear models21.3 Systems of equations21.4 Additional nonlinear least squares tools21.5 Nonnegative least squares21.6 Noisy objective functions21.7 Moving forwardReferences

Content preview from Nonlinear Parameter Optimization Using R Tools

Chapter 19Linking R to external optimization tools

This chapter is about using tools that are NOT in R but prepared in other computing systems. However, it is about using R to access these tools.

The issues in doing this are first of all technical issues. We must find a way to link our R session to the external tool in a way that the tool can be run successfully and we can get the results back in a form usable by R. There are several ways to do this, and the level of detail is such that here we will have to be satisfied with a limited treatment of the main ideas.

There are also legal issues that may concern us. R is free/libre software, and proprietary or otherwise limited licenses on the external tool mean that such tools cannot be distributed with R. Such issues occupy quite a large number of the contributions on the R mailing lists and to some extent fragment the R environment. Personally, I will only work with free/libre software unless there is some compelling reason to do otherwise. I am a scientist and want to share ideas with others, and this cannot be done properly unless source code and data are available so that computations can be reproduced.

Finally, there are personal preferences. My own choices are, as far as is reasonable, to avoid complexity that can give rise to errors. In any situation where one has to deal with more than one structure or system, such errors can be extremely difficult to debug.

Nevertheless, there are situations where it is necessary to use ...