Theory and Methods of Statistics

11.9 Some Alternate Methods for Regression

In a regression setting with p − 1 independent variables, in some cases, Y is related to only k < p − 1 independent variables, and thus it makes sense to remove some of the independent variables from the model. Stepwise regression, all subsets regression, or a penalty method such as lasso may be used for this purpose. In some cases, another situation may be true where Y is related to all the independent variables and the independent variables are well correlated among themselves. In such cases, it may be reasonable to regress Y on a few (say k < p − 1) appropriately created linear combinations of the independent variables. Partial least squares (PLS) or principal components regression (PCR) may be ...