Machine Learning Algorithms - Second Edition

Passive-aggressive algorithms

In this section, we are going to analyze an algorithm that can be efficiently employed for online linear classifications. In fact, one of the problems with other methods is that when new samples are collected, the whole model must be retrained. The main idea proposed by Crammer et al. (in Online Passive-Aggressive Algorithms, Crammer K., Dekel O., Keshet J., Shalev-Shwartz S., Singer Y., Journal of Machine Learning Research 7 (2006) 551–585) is to train a model incrementally, allowing modifications of the parameters only when needed, while discarding all the updates that don't alter the equilibrium. In the original paper, three variants were proposed. In this description, we are considering the one called PA-II ...