book

Machine Learning with Python for Everyone

by Mark Fenner

August 2019

Beginner to intermediate

353 pages

18h 48m

English

Addison-Wesley Professional

Read now

Unlock full access

Includes

Has Sandbox

AudienceApproachOverviewAcknowledgmentsPublisher’s Note

1.1 Welcome1.2 Scope, Terminology, Prediction, and Data1.3 Putting the Machine in Machine Learning1.4 Examples of Learning Systems1.5 Evaluating Learning Systems1.6 A Process for Building Learning Systems1.7 Assumptions and Reality of Learning1.8 End-of-Chapter Material
2.1 About Our Setup2.2 The Need for Mathematical Language2.3 Our Software for Tackling Machine Learning2.4 Probability2.5 Linear Combinations, Weighted Sums, and Dot Products2.6 A Geometric View: Points in Space2.7 Notation and the Plus-One Trick2.8 Getting Groovy, Breaking the Straight-Jacket, and Nonlinearity2.9 NumPy versus “All the Maths”2.10 Floating-Point Issues2.11 EOC
3.1 Classification Tasks3.2 A Simple Classification Dataset3.3 Training and Testing: Don’t Teach to the Test3.4 Evaluation: Grading the Exam3.5 Simple Classifier #1: Nearest Neighbors, Long Distance Relationships, and Assumptions3.6 Simple Classifier #2: Naive Bayes, Probability, and Broken Promises3.7 Simplistic Evaluation of Classifiers3.8 EOC
4.1 A Simple Regression Dataset4.2 Nearest-Neighbors Regression and Summary Statistics4.3 Linear Regression and Errors4.4 Optimization: Picking the Best Answer4.5 Simple Evaluation and Comparison of Regressors4.6 EOC
5.1 Evaluation and Why Less Is More5.2 Terminology for Learning Phases5.3 Major Tom, There’s Something Wrong: Overfitting and Underfitting5.4 From Errors to Costs5.5 (Re)Sampling: Making More from Less5.6 Break-It-Down: Deconstructing Error into Bias and Variance5.7 Graphical Evaluation and Comparison5.8 Comparing Learners with Cross-Validation5.9 EOC
6.1 Baseline Classifiers6.2 Beyond Accuracy: Metrics for Classification6.3 ROC Curves6.4 Another Take on Multiclass: One-versus-One6.5 Precision-Recall Curves6.6 Cumulative Response and Lift Curves6.7 More Sophisticated Evaluation of Classifiers: Take Two6.8 EOC
7.1 Baseline Regressors7.2 Additional Measures for Regression7.3 Residual Plots7.4 A First Look at Standardization7.5 Evaluating Regressors in a More Sophisticated Way: Take Two7.6 EOC
8.1 Revisiting Classification8.2 Decision Trees8.3 Support Vector Classifiers8.4 Logistic Regression8.5 Discriminant Analysis8.6 Assumptions, Biases, and Classifiers8.7 Comparison of Classifiers: Take Three8.8 EOC
9.1 Linear Regression in the Penalty Box: Regularization9.2 Support Vector Regression9.3 Piecewise Constant Regression9.4 Regression Trees9.5 Comparison of Regressors: Take Three9.6 EOC
10.1 Feature Engineering Terminology and Motivation10.2 Feature Selection and Data Reduction: Taking out the Trash10.3 Feature Scaling10.4 Discretization10.5 Categorical Coding10.6 Relationships and Interactions10.7 Target Manipulations10.8 EOC
11.1 Models, Parameters, Hyperparameters11.2 Tuning Hyperparameters11.3 Down the Recursive Rabbit Hole: Nested Cross-Validation11.4 Pipelines11.5 Pipelines and Tuning Together11.6 EOC
12.1 Ensembles12.2 Voting Ensembles12.3 Bagging and Random Forests12.4 Boosting12.5 Comparing the Tree-Ensemble Methods12.6 EOC
13.1 Feature Selection13.2 Feature Construction with Kernels13.3 Principal Components Analysis: An Unsupervised Technique13.4 EOC
14.1 Working with Text14.2 Clustering14.3 Working with Images14.4 EOC
15.1 Optimization15.2 Linear Regression from Raw Materials15.3 Building Logistic Regression from Raw Materials15.4 SVM from Raw Materials15.5 Neural Networks15.6 Probabilistic Graphical Models15.7 EOC

Content preview from Machine Learning with Python for Everyone

5. Evaluating and Comparing Learners

In [1]:

Click here to view code image

# setup 
from mlwpy import *
diabetes = datasets.load_diabetes()
%matplotlib inline

5.1 Evaluation and Why Less Is More

Lao Tzu: Those that know others are wise. Those that know themselves are Enlightened.

The biggest risk in developing a learning system is overestimating how well it will do when we use it. I touched on this risk in our first look at classification. Those of us that have studied for a test and thought we had a good mastery of the material, and then bombed the test, will be intimately familiar with this risk. It is very, very easy to (1) think we know a lot and will do well on an exam and (2) not do very well on the exam. On a test, we may discover we ...