Pattern Recognition, 4th Edition

4.19.2. Robust Statistics Regression

The regression task was introduced in Section 3.5.1. Let y ∈ R, x ∈ R^l be two statistically dependent random entities. Given a set of training samples (yⁱ, xⁱ), the goal is to compute a function g(x) that optimally estimates the value of y when x is measured. In a number of cases, mean square or least squares type of costs are not the most appropriate ones. For example, in cases where the statistical distribution of the data has long tails, then using the least squares criterion will lead to a solution dominated by a small number of points that have very large values (outliers). A similar situation can occur from incorrectly labeled data. Take, for example, a single training data point whose target value ...