Building Machine Learning Systems with Python - Third Edition

Bias, variance and their trade-off

In Chapter 1, Getting Started with Python Machine Learning, we tried to fit polynomials of different complexities controlled by the d dimensionality parameter to fit the data. We realized that a two-dimensional polynomial, a straight line, does not fit the example data very well, because the data was not linear in nature. No matter how elaborate our fitting procedure was, our two-dimensional model saw everything as a straight line. We learned that it was too biased for the data at hand and called it under-fitting.

We played a bit with the dimensions and found out that the 100-dimensional polynomial fits very well to the data on which it was trained (we did not know about train-test splits at that time). ...