book

The Data Science Workshop

Name: The Data Science Workshop
ISBN: 9781838981266

by Anthony So, Thomas Joseph, Robert Thas John, Andrew Worsley, Dr. Samuel Asare, Ivan Liu, Tiffany Ford, Barbora stetinova, Pritesh Tiwari

January 2020

Beginner

818 pages

15h 45m

English

Packt Publishing

Read now

Unlock full access

Preface
About the BookAbout the ChaptersConventionsBefore You BeginHow to Set Up Google ColabHow to Use Google ColabInstalling the Code Bundle
1. Introduction to Data Science in Python
IntroductionApplication of Data ScienceWhat Is Machine Learning?Supervised LearningUnsupervised LearningReinforcement LearningOverview of PythonTypes of VariableNumeric VariablesText VariablesPython ListPython DictionaryExercise 1.01: Creating a Dictionary That Will Contain Machine Learning AlgorithmsPython for Data ScienceThe pandas PackageDataFrame and SeriesCSV FilesExcel SpreadsheetsJSONExercise 1.02: Loading Data of Different Formats into a pandas DataFrameScikit-LearnWhat Is a Model?Model HyperparametersThe sklearn APIExercise 1.03: Predicting Breast Cancer from a Dataset Using sklearnActivity 1.01: Train a Spam Detector AlgorithmSummary
2. Regression
IntroductionSimple Linear RegressionThe Method of Least SquaresMultiple Linear RegressionEstimating the Regression Coefficients (β0, β1, β2 and β3)Logarithmic Transformations of VariablesCorrelation MatricesConducting Regression Analysis Using PythonExercise 2.01: Loading and Preparing the Data for AnalysisThe Correlation CoefficientExercise 2.02: Graphical Investigation of Linear Relationships Using PythonExercise 2.03: Examining a Possible Log-Linear Relationship Using PythonThe Statsmodels formula APIExercise 2.04: Fitting a Simple Linear Regression Model Using the Statsmodels formula APIAnalyzing the Model SummaryThe Model Formula LanguageIntercept HandlingActivity 2.01: Fitting a Log-Linear Model Using the Statsmodels formula APIMultiple Regression AnalysisExercise 2.05: Fitting a Multiple Linear Regression Model Using the Statsmodels formula APIAssumptions of Regression AnalysisActivity 2.02: Fitting a Multiple Log-Linear Regression ModelExplaining the Results of Regression AnalysisRegression Analysis Checks and BalancesThe F-testThe t-testSummary
3. Binary Classification
IntroductionUnderstanding the Business ContextBusiness DiscoveryExercise 3.01: Loading and Exploring the Data from the DatasetTesting Business Hypotheses Using Exploratory Data AnalysisVisualization for Exploratory Data AnalysisExercise 3.02: Business Hypothesis Testing for Age versus Propensity for a Term LoanIntuitions from the Exploratory AnalysisActivity 3.01: Business Hypothesis Testing to Find Employment Status versus Propensity for Term DepositsFeature Engineering Business-Driven Feature EngineeringExercise 3.03: Feature Engineering – Exploration of Individual FeaturesExercise 3.04: Feature Engineering – Creating New Features from Existing OnesData-Driven Feature EngineeringA Quick Peek at Data Types and a Descriptive SummaryCorrelation Matrix and VisualizationExercise 3.05: Finding the Correlation in Data to Generate a Correlation Plot Using Bank DataSkewness of DataHistogramsDensity PlotsOther Feature Engineering MethodsSummarizing Feature EngineeringBuilding a Binary Classification Model Using the Logistic Regression FunctionLogistic Regression DemystifiedMetrics for Evaluating Model PerformanceConfusion MatrixAccuracyClassification ReportData PreprocessingExercise 3.06: A Logistic Regression Model for Predicting the Propensity of Term Deposit Purchases in a BankActivity 3.02: Model Iteration 2 – Logistic Regression Model with Feature Engineered VariablesNext StepsSummary
4. Multiclass Classification with RandomForest
IntroductionTraining a Random Forest ClassifierEvaluating the Model's PerformanceExercise 4.01: Building a Model for Classifying Animal Type and Assessing Its PerformanceNumber of Trees EstimatorExercise 4.02: Tuning n_estimators to Reduce OverfittingMaximum DepthExercise 4.03: Tuning max_depth to Reduce OverfittingMinimum Sample in LeafExercise 4.04: Tuning min_samples_leafMaximum FeaturesExercise 4.05: Tuning max_featuresActivity 4.01: Train a Random Forest Classifier on the ISOLET DatasetSummary
5. Performing Your First Cluster Analysis
IntroductionClustering with k-meansExercise 5.01: Performing Your First Clustering Analysis on the ATO DatasetInterpreting k-means ResultsExercise 5.02: Clustering Australian Postcodes by Business Income and ExpensesChoosing the Number of ClustersExercise 5.03: Finding the Optimal Number of ClustersInitializing ClustersExercise 5.04: Using Different Initialization Parameters to Achieve a Suitable OutcomeCalculating the Distance to the CentroidExercise 5.05: Finding the Closest Centroids in Our DatasetStandardizing DataExercise 5.06: Standardizing the Data from Our DatasetActivity 5.01: Perform Customer Segmentation Analysis in a Bank Using k-meansSummary
6. How to Assess Performance
IntroductionSplitting DataExercise 6.01: Importing and Splitting DataAssessing Model Performance for Regression ModelsData Structures – Vectors and MatricesScalarsVectorsMatricesR2 ScoreExercise 6.02: Computing the R2 Score of a Linear Regression ModelMean Absolute ErrorExercise 6.03: Computing the MAE of a ModelExercise 6.04: Computing the Mean Absolute Error of a Second ModelOther Evaluation MetricsAssessing Model Performance for Classification ModelsExercise 6.05: Creating a Classification Model for Computing Evaluation MetricsThe Confusion MatrixExercise 6.06: Generating a Confusion Matrix for the Classification ModelMore on the Confusion Matrix PrecisionExercise 6.07: Computing Precision for the Classification ModelRecallExercise 6.08: Computing Recall for the Classification ModelF1 ScoreExercise 6.09: Computing the F1 Score for the Classification ModelAccuracyExercise 6.10: Computing Model Accuracy for the Classification ModelLogarithmic LossExercise 6.11: Computing the Log Loss for the Classification ModelReceiver Operating Characteristic CurveExercise 6.12: Computing and Plotting ROC Curve for a Binary Classification ProblemArea Under the ROC CurveExercise 6.13: Computing the ROC AUC for the Caesarian DatasetSaving and Loading ModelsExercise 6.14: Saving and Loading a ModelActivity 6.01: Train Three Different Models and Use Evaluation Metrics to Pick the Best Performing ModelSummary
7. The Generalization of Machine Learning Models
IntroductionOverfittingTraining on Too Many FeaturesTraining for Too LongUnderfittingDataThe Ratio for Dataset SplitsCreating Dataset SplitsExercise 7.01: Importing and Splitting DataRandom StateExercise 7.02: Setting a Random State When Splitting DataCross-ValidationKFoldExercise 7.03: Creating a Five-Fold Cross-Validation DatasetExercise 7.04: Creating a Five-Fold Cross-Validation Dataset Using a Loop for Callscross_val_scoreExercise 7.05: Getting the Scores from Five-Fold Cross-ValidationUnderstanding Estimators That Implement CVLogisticRegressionCVExercise 7.06: Training a Logistic Regression Model Using Cross-ValidationHyperparameter Tuning with GridSearchCVDecision TreesExercise 7.07: Using Grid Search with Cross-Validation to Find the Best Parameters for a ModelHyperparameter Tuning with RandomizedSearchCVExercise 7.08: Using Randomized Search for Hyperparameter TuningModel Regularization with Lasso RegressionExercise 7.09: Fixing Model Overfitting Using Lasso RegressionRidge RegressionExercise 7.10: Fixing Model Overfitting Using Ridge RegressionActivity 7.01: Find an Optimal Model for Predicting the Critical Temperatures of SuperconductorsSummary
8. Hyperparameter Tuning
IntroductionWhat Are Hyperparameters?Difference between Hyperparameters and Statistical Model ParametersSetting HyperparametersA Note on DefaultsFinding the Best HyperparameterizationExercise 8.01: Manual Hyperparameter Tuning for a k-NN ClassifierAdvantages and Disadvantages of a Manual SearchTuning Using Grid SearchSimple Demonstration of the Grid Search StrategyGridSearchCVTuning using GridSearchCVSupport Vector Machine (SVM) ClassifiersExercise 8.02: Grid Search Hyperparameter Tuning for an SVMAdvantages and Disadvantages of Grid SearchRandom SearchRandom Variables and Their DistributionsSimple Demonstration of the Random Search ProcessTuning Using RandomizedSearchCVExercise 8.03: Random Search Hyperparameter Tuning for a Random Forest ClassifierAdvantages and Disadvantages of a Random SearchActivity 8.01: Is the Mushroom Poisonous?Summary
9. Interpreting a Machine Learning Model
IntroductionLinear Model CoefficientsExercise 9.01: Extracting the Linear Regression Coefficient RandomForest Variable ImportanceExercise 9.02: Extracting RandomForest Feature ImportanceVariable Importance via PermutationExercise 9.03: Extracting Feature Importance via PermutationPartial Dependence PlotsExercise 9.04: Plotting Partial DependenceLocal Interpretation with LIME Exercise 9.05: Local Interpretation with LIMEActivity 9.01: Train and Analyze a Network Intrusion Detection ModelSummary

10. Analyzing a Dataset
IntroductionExploring Your DataAnalyzing Your DatasetExercise 10.01: Exploring the Ames Housing Dataset with Descriptive StatisticsAnalyzing the Content of a Categorical VariableExercise 10.02: Analyzing the Categorical Variables from the Ames Housing DatasetSummarizing Numerical VariablesExercise 10.03: Analyzing Numerical Variables from the Ames Housing DatasetVisualizing Your DataHow to use the Altair APIHistogram for Numerical VariablesBar Chart for Categorical Variables BoxplotsExercise 10.04: Visualizing the Ames Housing Dataset with AltairActivity 10.01: Analyzing Churn Data Using Visual Data Analysis TechniquesSummary
11. Data Preparation
IntroductionHandling Row DuplicationExercise 11.01: Handling Duplicates in a Breast Cancer DatasetConverting Data TypesExercise 11.02: Converting Data Types for the Ames Housing DatasetHandling Incorrect ValuesExercise 11.03: Fixing Incorrect Values in the State ColumnHandling Missing ValuesExercise 11.04: Fixing Missing Values for the Horse Colic DatasetActivity 11.01: Preparing the Speed Dating DatasetSummary
12. Feature Engineering
IntroductionMerging DatasetsThe left joinThe right joinExercise 12.01: Merging the ATO Dataset with the Postcode DataBinning VariablesExercise 12.02: Binning the YearBuilt variable from the AMES Housing datasetManipulating DatesExercise 12.03: Date Manipulation on Financial Services Consumer ComplaintsPerforming Data AggregationExercise 12.04: Feature Engineering Using Data Aggregation on the AMES Housing DatasetActivity 12.01: Feature Engineering on a Financial DatasetSummary
13. Imbalanced Datasets
IntroductionUnderstanding the Business ContextExercise 13.01: Benchmarking the Logistic Regression Model on the DatasetAnalysis of the ResultChallenges of Imbalanced DatasetsStrategies for Dealing with Imbalanced DatasetsCollecting More DataResampling DataExercise 13.02: Implementing Random Undersampling and Classification on Our Banking Dataset to Find the Optimal ResultAnalysisGenerating Synthetic SamplesImplementation of SMOTE and MSMOTEExercise 13.03: Implementing SMOTE on Our Banking Dataset to Find the Optimal ResultExercise 13.04: Implementing MSMOTE on Our Banking Dataset to Find the Optimal ResultApplying Balancing Techniques on a Telecom DatasetActivity 13.01: Finding the Best Balancing Technique by Fitting a Classifier on the Telecom Churn DatasetSummary
14. Dimensionality Reduction
IntroductionBusiness ContextExercise 14.01: Loading and Cleaning the DatasetCreating a High-Dimensional DatasetActivity 14.01: Fitting a Logistic Regression Model on a High‑Dimensional DatasetStrategies for Addressing High-Dimensional DatasetsBackward Feature Elimination (Recursive Feature Elimination)Exercise 14.02: Dimensionality Reduction Using Backward Feature EliminationForward Feature SelectionExercise 14.03: Dimensionality Reduction Using Forward Feature SelectionPrincipal Component Analysis (PCA)Exercise 14.04: Dimensionality Reduction Using PCAIndependent Component Analysis (ICA)Exercise 14.05: Dimensionality Reduction Using Independent Component AnalysisFactor AnalysisExercise 14.06: Dimensionality Reduction Using Factor AnalysisComparing Different Dimensionality Reduction TechniquesActivity 14.02: Comparison of Dimensionality Reduction Techniques on the Enhanced Ads DatasetSummary
15. Ensemble Learning
IntroductionEnsemble LearningVarianceBiasBusiness ContextExercise 15.01: Loading, Exploring, and Cleaning the DataActivity 15.01: Fitting a Logistic Regression Model on Credit Card DataSimple Methods for Ensemble LearningAveragingExercise 15.02: Ensemble Model Using the Averaging TechniqueWeighted AveragingExercise 15.03: Ensemble Model Using the Weighted Averaging TechniqueIteration 2 with Different WeightsMax VotingExercise 15.04: Ensemble Model Using Max VotingAdvanced Techniques for Ensemble LearningBaggingExercise 15.05: Ensemble Learning Using BaggingBoostingExercise 15.06: Ensemble Learning Using BoostingStackingExercise 15.07: Ensemble Learning Using StackingActivity 15.02: Comparison of Advanced Ensemble TechniquesSummary
16. Machine Learning Pipelines
IntroductionPipelinesBusiness ContextExercise 16.01: Preparing the Dataset to Implement PipelinesAutomating ML Workflows Using PipelineAutomating Data Preprocessing Using PipelinesExercise 16.02: Applying Pipelines for Feature Extraction to the DatasetML Pipeline with Processing and Dimensionality ReductionExercise 16.03: Adding Dimensionality Reduction to the Feature Extraction PipelineML Pipeline for Modeling and PredictionExercise 16.04: Modeling and Predictions Using ML PipelinesML Pipeline for Spot-Checking Multiple ModelsExercise 16.05: Spot-Checking Models Using ML PipelinesML Pipelines for Identifying the Best Parameters for a ModelCross-ValidationGrid SearchExercise 16.06: Grid Search and Cross-Validation with ML PipelinesApplying Pipelines to a DatasetActivity 16.01: Complete ML Workflow in a PipelineSummary
17. Automated Feature Engineering
IntroductionFeature EngineeringAutomating Feature Engineering Using Feature ToolsBusiness ContextDomain Story for the Problem StatementFeaturetools – Creating Entities and RelationshipsExercise 17.01: Defining Entities and Establishing RelationshipsFeature Engineering – Basic OperationsFeaturetools – Automated Feature EngineeringExercise 17.02: Creating New Features Using Deep Feature SynthesisExercise 17.03: Classification Model after Automated Feature GenerationFeaturetools on a New DatasetActivity 17.01: Building a Classification Model with Features that have been Generated Using FeaturetoolsSummary

Content preview from The Data Science Workshop

6. How to Assess Performance

Overview

This chapter will introduce you to model evaluation, where you evaluate or assess the performance of each model that you train before you decide to put it into production. By the end of this chapter, you will be able to create an evaluation dataset. You will be equipped to assess the performance of linear regression models using mean absolute error (MAE) and mean squared error (MSE). You will also be able to evaluate the performance of logistic regression models using accuracy, precision, recall, and F1 score.

Introduction

When you assess the performance of a model, you look at certain measurements or values that tell you how well the model is performing under certain conditions, and that helps you ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

The Data Science Workshop - Second Edition

Publisher Resources

ISBN: 9781838981266

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

The Data Science Workshop

by Anthony So, Thomas Joseph, Robert Thas John, Andrew Worsley, Dr. Samuel Asare, Ivan Liu, Tiffany Ford, Barbora stetinova, Pritesh Tiwari

6. How to Assess Performance

Introduction

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.