Skip to Main Content
Data Science Using Python and R
book

Data Science Using Python and R

by Chantal D. Larose, Daniel T. Larose
April 2019
Beginner to intermediate content levelBeginner to intermediate
240 pages
6h 47m
English
Wiley
Content preview from Data Science Using Python and R

Chapter 7MODEL EVALUATION

7.1 INTRODUCTION TO MODEL EVALUATION

So far in Data Science Using Python and R, we have covered the first five phases of the Data Science Methodology:

  1. Data Understanding Phase
  2. Data Preparation Phase
  3. Exploratory Data Analysis Phase
  4. Setup Phase
  5. Modeling Phase (at least a little bit)

But, so far we have not examined whether our models are any good. That is, we have not evaluated their usefulness in making predictions. Note the difference between evaluation and validation. Model validation simply makes sure that our model results are consistent between the training and test data sets. But, model validation does not tell us how accurate our models are, or what their error rate is. For measures like these, we need to turn to model evaluation. Since the only models we have learned so far are decision trees for classification, we shall restrict our discussion to evaluative measures for classification models.

7.2 CLASSIFICATION EVALUATION MEASURES

We will develop classification evaluation measures for the case where we have a binary target variable. In order to apply the measures we will learn in this chapter, we will need to denote (arbitrarily, if desired) one of the two target outcomes as positive and one as negative. For example, suppose we are trying to predict income, a binary variable with values high income and low income. We could denote high income as positive and low income as negative.1

Now, the classification model evaluation measures we will ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Start your free trial

You might also like

Practical Data Science with Python 3: Synthesizing Actionable Insights from Data

Practical Data Science with Python 3: Synthesizing Actionable Insights from Data

Ervin Varga
Python Data Science Essentials - Third Edition

Python Data Science Essentials - Third Edition

Alberto Boschetti, Luca Massaron, Pietro Marinelli, Matteo Malosetti

Publisher Resources

ISBN: 9781119526810Purchase book