Engineering Optimization

29Regression

29.1 Introduction

This chapter is a summary of key issues and solutions related to nonlinear regression–fitting nonlinear models to data. The details of diverse issues are revealed in the book Nonlinear Regression Modeling for Engineering Applications: Modeling, Model Validation, and Enabling Design of Experiments by Rhinehart, R. R., John Wiley & Sons, Inc., Hoboken, NJ, 2016b. Here, they are summarized and presented with an optimization application perspective.

Regression is the procedure of fitting a model to data and gives rise to the following optimization questions:

1. Classic nonlinear regression procedures direct the user to transform the model so that linear regression can be applied. Should one?
2. In regression with dynamic models, it is not uncommon to have a delay, an integer multiple of time‐sampling intervals. If delay is a DV, then it is a discretized variable. Compounding the associated difficulties, in regression of nonlinear models, the continuum variables can be constrained, and the models have discontinuities. What optimization method is best?
3. Classically, regression minimizes the squared deviation between model and experimental response, the vertical deviation. This presumes that there is no uncertainty on the input variables and that the y‐variance is uniform over all data. Often these are not reasonable assumptions, and maximum likelihood is a better concept to use. But maximum likelihood is computationally complex. Is there a useful compromise ...