40 Algorithms Every Programmer Should Know

Bias-variance trade-off

When training a particular machine learning model, it is tricky to decide the right level of generalization for the rules that comprise a trained model. The struggle to come up with the right level of generalization is captured by the bias-variance trade-off.

Note that more simplistic assumptions = more generalization = low variance = high variance.

This trade-off between bias and variance is determined by the choice of algorithm, the characteristics of the data, and various hyperparameters. It is important to achieve the right compromise between the bias and variance based on the requirements of the specific problem you are trying to solve.