Machine Learning in Elixir

Regularizing to Generalize

As you learned in the previous section, the difference between machine learning and optimization is that machine learning is concerned with performance on unseen data. In Chapter 1, ​Make Machines That Learn​, you learned that this concept is known as generalization. Your objective is to train models that generalize—performance on training data isn’t necessarily important.

Imagine you have two models. Model A performs noticeably worse on the training set than model B but noticeably better on the test set than model B. In this scenario, which model would you prefer?

Given your primary objective is performance on unseen data, for example, the test set, you’d likely prefer model B from a model performance perspective. ...