Nonlinear Parameter Optimization Using R Tools

Nonlinear Parameter Optimization Using R Tools

Released May 2014
Publisher(s): Wiley
ISBN: 9781118569283

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

Nonlinear Parameter Optimization Using R

John C. Nash, Telfer School of Management, University of Ottawa, Canada

A systematic and comprehensive treatment of optimization software using R

In recent decades, optimization techniques have been streamlined by computational and artificial intelligence methods to analyze more variables, especially under non-linear, multivariable conditions, more quickly than ever before.

Optimization is an important tool for decision science and for the analysis of physical systems used in engineering. Nonlinear Parameter Optimization with R explores the principal tools available in R for function minimization, optimization, and nonlinear parameter determination and features numerous examples throughout.

Nonlinear Parameter Optimization with R:

  • Provides a comprehensive treatment of optimization techniques

  • Examines optimization problems that arise in statistics and how to solve them using R

  • Enables researchers and practitioners to solve parameter determination problems

  • Presents traditional methods as well as recent developments in R

  • Is supported by an accompanying website featuring R code, examples and datasets

    • Researchers and practitioners who have to solve parameter determination problems who are users of R but are novices in the field optimization or function minimization will benefit from this book. It will also be useful for scientists building and estimating nonlinear models in various fields such as hydrology, sports forecasting, ecology, chemical engineering, pharmaco-kinetics, agriculture, economics and statistics.

    Table of contents

    1. Cover
    2. Title Page
    3. Copyright
    4. Dedication
    5. Preface
    6. Chapter 1: Optimization problem tasks and how they arise
      1. 1.1 The general optimization problem
      2. 1.2 Why the general problem is generally uninteresting
      3. 1.3 (Non-)Linearity
      4. 1.4 Objective function properties
      5. 1.5 Constraint types
      6. 1.6 Solving sets of equations
      7. 1.7 Conditions for optimality
      8. 1.8 Other classifications
      9. References
    7. Chapter 2: Optimization algorithms—an overview
      1. 2.1 Methods that use the gradient
      2. 2.2 Newton-like methods
      3. 2.3 The promise of Newton's method
      4. 2.4 Caution: convergence versus termination
      5. 2.5 Difficulties with Newton's method
      6. 2.6 Least squares: Gauss–Newton methods
      7. 2.7 Quasi-Newton or variable metric method
      8. 2.8 Conjugate gradient and related methods
      9. 2.9 Other gradient methods
      10. 2.10 Derivative-free methods
      11. 2.11 Stochastic methods
      12. 2.12 Constraint-based methods—mathematical programming
      13. References
    8. Chapter 3: Software structure and interfaces
      1. 3.1 Perspective
      2. 3.2 Issues of choice
      3. 3.3 Software issues
      4. 3.4 Specifying the objective and constraints to the optimizer
      5. 3.5 Communicating exogenous data to problem definition functions
      6. 3.6 Masked (temporarily fixed) optimization parameters
      7. 3.7 Dealing with inadmissible results
      8. 3.8 Providing derivatives for functions
      9. 3.9 Derivative approximations when there are constraints
      10. 3.10 Scaling of parameters and function
      11. 3.11 Normal ending of computations
      12. 3.12 Termination tests—abnormal ending
      13. 3.13 Output to monitor progress of calculations
      14. 3.14 Output of the optimization results
      15. 3.15 Controls for the optimizer
      16. 3.16 Default control settings
      17. 3.17 Measuring performance
      18. 3.18 The optimization interface
      19. References
    9. Chapter 4: One-parameter root-finding problems
      1. 4.1 Roots
      2. 4.2 Equations in one variable
      3. 4.3 Some examples
      4. 4.4 Approaches to solving 1D root-finding problems
      5. 4.5 What can go wrong?
      6. 4.6 Being a smart user of root-finding programs
      7. 4.7 Conclusions and extensions
      8. References
    10. Chapter 5: One-parameter minimization problems
      1. 5.1 The optimize() function
      2. 5.2 Using a root-finder
      3. 5.3 But where is the minimum?
      4. 5.4 Ideas for 1D minimizers
      5. 5.5 The line-search subproblem
      6. References
    11. Chapter 6: Nonlinear least squares
      1. 6.1 nls() from package stats
      2. 6.2 A more difficult case
      3. 6.3 The structure of the nls() solution
      4. 6.4 Concerns with nls()
      5. 6.5 Some ancillary tools for nonlinear least squares
      6. 6.6 Minimizing R functions that compute sums of squares
      7. 6.7 Choosing an approach
      8. 6.8 Separable sums of squares problems
      9. 6.9 Strategies for nonlinear least squares
      10. References
    12. Chapter 7: Nonlinear equations
      1. 7.1 Packages and methods for nonlinear equations
      2. 7.2 A simple example to compare approaches
      3. 7.3 A statistical example
      4. References
    13. Chapter 8: Function minimization tools in the base R system
      1. 8.1 optim()
      2. 8.2 nlm()
      3. 8.3 nlminb()
      4. 8.4 Using the base optimization tools
      5. References
    14. Chapter 9: Add-in function minimization packages for R
      1. 9.1 Package optimx
      2. 9.2 Some other function minimization packages
      3. 9.3 Should we replace optim() routines?
      4. References
    15. Chapter 10: Calculating and using derivatives
      1. 10.1 Why and how
      2. 10.2 Analytic derivatives—by hand
      3. 10.3 Analytic derivatives—tools
      4. 10.4 Examples of use of R tools for differentiation
      5. 10.5 Simple numerical derivatives
      6. 10.6 Improved numerical derivative approximations
      7. 10.7 Strategy and tactics for derivatives
      8. References
    16. Chapter 11: Bounds constraints
      1. 11.1 Single bound: use of a logarithmic transformation
      2. 11.2 Interval bounds: Use of a hyperbolic transformation
      3. 11.3 Setting the objective large when bounds are violated
      4. 11.4 An active set approach
      5. 11.5 Checking bounds
      6. 11.6 The importance of using bounds intelligently
      7. 11.7 Post-solution information for bounded problems
      8. Appendix 11.A Function transfinite
      9. References
    17. Chapter 12: Using masks
      1. 12.1 An example
      2. 12.2 Specifying the objective
      3. 12.3 Masks for nonlinear least squares
      4. 12.4 Other approaches to masks
      5. References
    18. Chapter 13: Handling general constraints
      1. 13.1 Equality constraints
      2. 13.2 Sumscale problems
      3. 13.3 Inequality constraints
      4. 13.4 A perspective on penalty function ideas
      5. 13.5 Assessment
      6. References
    19. Chapter 14: Applications of mathematical programming
      1. 14.1 Statistical applications of math programming
      2. 14.2 R packages for math programming
      3. 14.3 Example problem: L1 regression
      4. 14.4 Example problem: minimax regression
      5. 14.5 Nonlinear quantile regression
      6. 14.6 Polynomial approximation
      7. References
    20. Chapter 15: Global optimization and stochastic methods
      1. 15.1 Panorama of methods
      2. 15.2 R packages for global and stochastic optimization
      3. 15.3 An example problem
      4. 15.4 Multiple starting values
      5. References
    21. Chapter 16: Scaling and reparameterization
      1. 16.1 Why scale or reparameterize?
      2. 16.2 Formalities of scaling and reparameterization
      3. 16.3 Hobbs' weed infestation example
      4. 16.4 The KKT conditions and scaling
      5. 16.5 Reparameterization of the weeds problem
      6. 16.6 Scale change across the parameter space
      7. 16.7 Robustness of methods to starting points
      8. 16.8 Strategies for scaling
      9. References
    22. Chapter 17: Finding the right solution
      1. 17.1 Particular requirements
      2. 17.2 Starting values for iterative methods
      3. 17.3 KKT conditions
      4. 17.4 Search tests
      5. References
    23. Chapter 18: Tuning and terminating methods
      1. 18.1 Timing and profiling
      2. 18.2 Profiling
      3. 18.3 More speedups of R computations
      4. 18.4 External language compiled functions
      5. 18.5 Deciding when we are finished
      6. References
    24. Chapter 19: Linking R to external optimization tools
      1. 19.1 Mechanisms to link R to external software
      2. 19.2 Prepackaged links to external optimization tools
      3. 19.3 Strategy for using external tools
      4. References
    25. Chapter 20: Differential equation models
      1. 20.1 The model
      2. 20.2 Background
      3. 20.3 The likelihood function
      4. 20.4 A first try at minimization
      5. 20.5 Attempts with optimx
      6. 20.6 Using nonlinear least squares
      7. 20.7 Commentary
      8. Reference
    26. Chapter 21: Miscellaneous nonlinear estimation tools for R
      1. 21.1 Maximum likelihood
      2. 21.2 Generalized nonlinear models
      3. 21.3 Systems of equations
      4. 21.4 Additional nonlinear least squares tools
      5. 21.5 Nonnegative least squares
      6. 21.6 Noisy objective functions
      7. 21.7 Moving forward
      8. References
    27. Appendix A: R packages used in examples
    28. Index
    29. End User License Agreement

    Product information

    • Title: Nonlinear Parameter Optimization Using R Tools
    • Author(s):
    • Release date: May 2014
    • Publisher(s): Wiley
    • ISBN: 9781118569283