Discovering Knowledge in Data: An Introduction to Data Mining, 2nd Edition

Chapter 14Model Evaluation Techniques

1. 14.1 Model Evaluation Techniques for the Description Task
2. 14.2 Model Evaluation Techniques for the Estimation and Prediction Tasks
3. 14.3 Model Evaluation Techniques for the Classification Task
4. 14.4 Error Rate, False Positives, and False Negatives
5. 14.5 Sensitivity and Specificity
6. 14.6 Misclassification Cost Adjustment to Reflect Real-World Concerns
7. 14.7 Decision Cost/Benefit Analysis
8. 14.8 Lift Charts and Gains Charts
9. 14.9 Interweaving Model Evaluation with Model Building
10. 14.10 Confluence of Results: Applying a Suite of Models
    1. The R Zone
    2. Reference
    3. Exercises
    4. Hands-On Analysis

As you may recall from Chapter 1, the CRISP cross-industry standard process for data mining consists of six phases, to be applied in an iterative cycle:

1. Business understanding phase
2. Data understanding phase
3. Data preparation phase
4. Modeling phase
5. Evaluation phase
6. Deployment phase

Nestled between the modeling and deployment phases comes the crucial evaluation phase, techniques for which are discussed in this chapter. By the time we arrive at the evaluation phase, the modeling phase has already generated one or more candidate models. It is of critical importance that these models be evaluated for quality and effectiveness before they are deployed for use in the field. Deployment of data mining models usually represents a capital expenditure and investment on the part of the company. If the models in question are invalid, the company's time and money are wasted. In this ...