Theory of Ridge Regression Estimation with Applications

11High‐Dimensional Ridge Regression

In biological, medical, bioinformatics, chemometrics, and many other fields, we involve high‐dimensional data, where the number of features (variables ) is (much) larger than the number of samples . In DNA microarray studies, e.g. identifying a set of candidate genes that are most likely related to the outcome in the experiment, we analyze thousands of genes simultaneously, while a limited number of samples are available. In computer vision and human face recognition, eigenvectors (known as eigenfaces) over the high‐dimensional vector space are used to make a low‐dimensional representation of face images. In functional magnetic resonance images (fMRI), we deal with hundreds of thousands of measurements (volumetric elements or “voxels” within the brain) sampled at hundreds of time points. Other examples include spatiotemporal data, financial data, ecological data, and so on. In such studies, traditional statistical methods fail to be applied. Hence, developing new statistical methods that can deal with the cases where the number of unknown parameters is much larger than the sample size, is of interest.

In the context of multiple linear models, it is challenging to have a least squares estimator (LSE) in high dimension. As it is outlined in Section ...