Introduction to Linear Regression Analysis, 5th Edition

CHAPTER 15

OTHER TOPICS IN THE USE OF REGRESSION ANALYSIS

This chapter surveys a variety of topics that arise in the use of regression analysis. In several cases only a brief glimpse of the subject is given along with references to more complete presentations.

15.1 ROBUST REGRESSION

15.1.1 Need for Robust Regression

When the observations y in the linear regression model y = Xβ + ε are normally distributed, the method of least squares is a good parameter estimation procedure in the sense that it produces an estimator of the parameter vector β that has good statistical properties. However, there are many situations where we have evidence that the distribution of the response variable is (considerably) nonnormal and/or there are outliers that affect the regression model. A case of considerable practical interest is one in which the observations follow a distribution that has longer or heavier tails than the normal. These heavy-tailed distributions tend to generate outliers, and these outliers may have a strong influence on the method of least squares in the sense that they “pull” the regression equation too much in their direction.

For example, consider the 10 observations shown in Figure 15.1 The point labeled A in this figure is just at the right end of the x space, but it has a response value that is near the average of the other 9 responses. If all the observations are considered, the resulting regression model is ŷ = 2.12 + 0.971x, and R² = 0.526. However, if we fit the linear ...