Credibility
Evaluating what’s been learned
Abstract
The success of machine learning in practical applications hinges on proper evaluation. This section discusses how the quality of predictions can be measured reliably. We consider the basic train-test setup for estimating predictive accuracy, before moving on to more sophisticated variants known as “cross-validation” and the “bootstrap” method. We also discuss the importance of proper parameter tuning when applying and evaluating machine learning, and explain how to use statistical significance tests when comparing the performance of two learning algorithms in a particular application domain. As well as basic classification accuracy, we consider other measures for evaluating the quality ...
Get Data Mining, 4th Edition now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
