5. Evaluating and Comparing Learners
# setup from mlwpy import * diabetes = datasets.load_diabetes() %matplotlib inline
5.1 Evaluation and Why Less Is More
Lao Tzu: Those that know others are wise. Those that know themselves are Enlightened.
The biggest risk in developing a learning system is overestimating how well it will do when we use it. I touched on this risk in our first look at classification. Those of us that have studied for a test and thought we had a good mastery of the material, and then bombed the test, will be intimately familiar with this risk. It is very, very easy to (1) think we know a lot and will do well on an exam and (2) not do very well on the exam. On a test, we may discover we ...