Machine Learning Algorithms - Second Edition

Huber regression

An alternative approach is based on a slightly modified loss function, called Huber loss (for a single sample):

The parameter t_H (called epsilon in scikit-learn) defines a threshold (based on the distance between target and prediction) that makes the loss function switch from a squared error to an absolute one. In this way, the magnitude of the loss changes accordingly, passing from a quadratic behavior to a linear one when the points are supposed to be outliers. In this way, their contribution to the global cost function is reduced and the hyperplane will remain closer to the majority of points even in presence of outliers. ...