Hands-On Ensemble Learning with Python

Trade-off

Bias and variance are two of the three major components that comprise a model's error. The third is called the irreducible error and can be attributed to inherent randomness or variability in the data. The total error of a model can be decomposed as follows:

As we saw earlier, bias and variance stem from the same source: model complexity. While bias arises from too little complexity and freedom, variance thrives in complex models. Thus, it is not possible to reduce bias without increasing variance and vice versa. Nevertheless, there is an optimal point of complexity, where the error is minimized as bias and variance are at an optimal ...