Hands-On Machine Learning for Algorithmic Trading

Underfitting versus overfitting

The following diagram illustrates overfitting by approximating a cosine function using increasingly complex polynomials and measuring the in-sample error. More specifically, we draw 10 random samples with some added noise (n = 30) to learn a polynomial of varying complexity (see the code in the accompanying notebook). Each time, the model predicts new data points and we capture the mean-squared error for these predictions, as well as the standard deviation of these errors.

The left-hand panel in the following diagram shows a polynomial of degree 1; a straight line clearly underfits the true function. However, the estimated line will not differ dramatically from one sample drawn from the true function to the ...