Python Machine Learning By Example - Second Edition

Overfitting, underfitting, and the bias-variance trade-off

Overfitting is a very important concept, hence, we're discussing it here, early in this book.

If we go through many practice questions for an exam, we may start to find ways to answer questions that have nothing to do with the subject material. For instance, given only five practice questions, we find that if there are two occurrences of potatoes, one tomato, and three occurrences of banana in a question, the answer is always A and if there is one potato, three occurrences of tomato, and two occurrences of banana in a question, the answer is always B, then we conclude this is always true and apply such a theory later on, even though the subject or answer may not be relevant to potatoes, ...