Machine Learning for OpenCV 4 - Second Edition

Applying Lasso and ridge regression

A common problem in machine learning is that an algorithm might work really well on the training set but, when applied to unseen data, it makes a lot of mistakes. You can see how this is problematic since, often, we are most interested in how a model generalizes to new data. Some algorithms (such as decision trees) are more susceptible to this phenomenon than others, but even linear regression can be affected.

This phenomenon is also known as overfitting, and we will talk about it extensively in Chapter 5, Using Decision Trees to Make a Medical Diagnosis, and Chapter 11, Selecting the Right Model with Hyperparameter Tuning.

A common technique for reducing overfitting is called regularization, which involves ...