Theory of Ridge Regression Estimation with Applications

6Ridge Regression in Theory and Applications

The multiple linear regression model is one of the best known and widely used among the models for statistical data analysis in every field of sciences and engineering as well as in social sciences, economics, and finance. The subject of this chapter is the study of the rigid regression estimator (RRE) for the regression coefficients, its characteristic properties, and comparing its relation with the least absolute shrinkage and selection operator (LASSO). Further, we consider the preliminary test estimator (PTE) and the Stein‐type ridge estimator in low dimension and study their dominance properties. We conclude the chapter with the asymptotic distributional theory of the ridge estimators following Knight and Fu (2000).

6.1 Multiple Linear Model Specification

Consider the multiple linear model with coefficient vector, given by

(6.1)

where is a vector of responses, is an design matrix of rank , and is an ‐vector of independently and ...