Machine Learning Algorithms - Second Edition

Passive-aggressive regression

We discussed the most common regression algorithms in Chapter 4, Regression Algorithms; however, the passive-aggressive strategy can also be employed to obtain efficient step-wise regression algorithms. We are not going to discuss all theoretical details (which can be found in the aforementioned paper), but it's helpful to introduce the ε-insensitive loss:

This loss function is analogous to a standard hinge loss, but it has been designed to work with continuous data. The role of parameter ε is to allow a low tolerance of prediction errors. In fact the following conditions hold:

In the first case, the prediction ...