Hands-On Ensemble Learning with Python

What is bias?

Bias refers to the inability of a method to correctly estimate the target. This does not only apply to machine learning. For example, in statistics, if we want to measure a population's average and do not sample carefully, the estimated average will be biased. In simple terms, the method's (sampling) estimation will not closely match the actual target (average).

In machine learning, bias refers to the difference between the expected prediction and its target. Biased models cannot properly fit the training data, resulting in poor in-sample performance and out-of-sample performance. A good example of a biased model arises when we try to fit a sine function with a simple linear regression. The model cannot fit the sine function, ...