book

Debugging Machine Learning Models with Python

Name: Debugging Machine Learning Models with Python
Author: Ali Madani
ISBN: 9781800208582

by Ali Madani

September 2023

Intermediate to advanced

344 pages

8h 50m

English

Packt Publishing

Read now

Unlock full access

Debugging Machine Learning Models with Python
ForewordContributorsAbout the authorAbout the reviewers
Preface
Who this book is forWhat this book coversTo get the most out of this bookDownload the example code filesConventions usedGet in touchShare your thoughtsDownload a free PDF copy of this book
Part 1:Debugging for Machine Learning Modeling
Chapter 1: Beyond Code Debugging
Technical requirementsMachine learning at a glanceTypes of machine learning modelingSupervised learningUnsupervised learningSelf-supervised learningSemi-supervised learningReinforcement learningGenerative machine learningDebugging in software developmentError messages in PythonDebugging techniquesDebuggersBest practices for high-quality Python programmingVersion controlDebugging beyond PythonFlaws in data used for modelingData format and structureData quantity and qualityData biasesModel and prediction-centric debuggingUnderfitting and overfittingInference in model testing and productionData or hyperparameters for changing landscapesSummaryQuestionsReferences
Chapter 2: Machine Learning Life Cycle
Technical requirementsBefore we start modelingData collectionData selectionData explorationData wranglingStructuringEnrichingData transformationCleaningModeling data preparationFeature selection and extractionDesigning an evaluation and testing strategyModel training and evaluationTesting the code and the modelModel deployment and monitoringSummaryQuestionsReferences
Chapter 3: Debugging toward Responsible AI
Technical requirementsImpartial modeling fairness in machine learningData biasAlgorithmic biasSecurity and privacy in machine learningData privacyData poisoningAdversarial attacksOutput integrity attacksSystem manipulationSecure and private machine learning techniquesTransparency in machine learning modelingAccountable and open to inspection modelingData and model governanceSummaryQuestionsReferences
Part 2:Improving Machine Learning Models
Chapter 4: Detecting Performance and Efficiency Issues in Machine Learning Models
Technical requirementsPerformance and error assessment measuresClassificationRegressionClusteringVisualization for performance assessmentSummary metrics are not enoughVisualizations could be misleadingDon’t interpret your plots as you wishBias and variance diagnosisModel validation strategyError analysisBeyond performanceSummaryQuestionsReferences
Chapter 5: Improving the Performance of Machine Learning Models
Technical requirementsOptions for improving model performanceGrid searchRandom searchBayesian searchSuccessive halvingSynthetic data generationOversampling for imbalanced dataImproving pre-training data processingAnomaly detection and outlier removalBenefitting from data of lower quality or relevanceRegularization to improve model generalizabilitySummaryQuestionsReferences
Chapter 6: Interpretability and Explainability in Machine Learning Modeling
Technical requirementsInterpretable versus black-box machine learningInterpretable machine learning modelsExplainability for complex modelsExplainability methods in machine learningLocal explainability techniquesGlobal explanationPracticing machine learning explainability in PythonExplanations in SHAPExplanations using LIMECounterfactual generation using Diverse Counterfactual Explanations (DiCE)Reviewing why having explainability is not enoughSummaryQuestionsReferences

Chapter 7: Decreasing Bias and Achieving Fairness
Technical requirementsFairness in machine learning modelingProxies for sensitive variablesSources of biasBiases introduced in data generation and collectionBias in model training and testingBias in productionUsing explainability techniquesFairness assessment and improvement in PythonSummaryQuestionsReferences
Part 3:Low-Bug Machine Learning Development and Deployment
Chapter 8: Controlling Risks Using Test-Driven Development
Technical requirementsTest-driven development for machine learning modelingUnit testingMachine learning differential testingTracking machine learning experimentsSummaryQuestionsReferences
Chapter 9: Testing and Debugging for Production
Technical requirementsInfrastructure testingInfrastructure as Code toolsInfrastructure testing toolsInfrastructure testing using PytestIntegration testing of machine learning pipelinesIntegration testing using pytestMonitoring and validating live performanceModel assertionSummaryQuestionsReferences
Chapter 10: Versioning and Reproducible Machine Learning Modeling
Technical requirementsReproducibility in machine learningData versioningModel versioningSummaryQuestionsReferences
Chapter 11: Avoiding and Detecting Data and Concept Drifts
Technical requirementsAvoiding drifts in your modelsAvoiding data driftAddressing concept driftDetecting driftsPracticing with alibi_detect for drift detectionPracticing with evidently for drift detectionSummaryQuestionsReferences
Part 4:Deep Learning Modeling
Chapter 12: Going Beyond ML Debugging with Deep Learning
Technical requirementsIntroduction to artificial neural networksOptimization algorithmsFrameworks for neural network modelingPyTorch for deep learning modelingSummaryQuestionsReferences
Chapter 13: Advanced Deep Learning Techniques
Technical requirementsTypes of neural networksCategorization based on data typeConvolutional neural networks for image shape dataPerformance assessmentCNN modeling using PyTorchImage data transformation and augmentation for CNNsUsing pre-trained modelsTransformers for language modelingTokenizationLanguage embeddingLanguage modeling using pre-trained modelsModeling graphs using deep neural networksGraph neural networksGNNs with PyTorch GeometricSummaryQuestionsReferences
Chapter 14: Introduction to Recent Advancements in Machine Learning
Technical requirementsGenerative modelingGenerative deep learning techniquesPrompt engineering for text-based generative modelsGenerative modeling using PyTorchReinforcement learningReinforcement learning with human feedback (RLHF)Self-supervised learning (SSL)Self-supervised learning with PyTorchSummaryQuestionsReferences
Part 5:Advanced Topics in Model Debugging
Chapter 15: Correlation versus Causality
Technical requirementsCorrelation as part of machine learning modelsCausal modeling to reduce risks and improve performanceAssessing causation in machine learning modelsCausal inferenceCausal modeling using PythonUsing dowhy for causal effect estimationUsing bnlearn for causal inference through Bayesian networksSummaryQuestionsReferences
Chapter 16: Security and Privacy in Machine Learning
Technical requirementsEncryption techniques and their use in machine learningImplementing AES encryption in PythonHomomorphic encryptionDifferential privacyFederated learningSummaryQuestionsReferences
Chapter 17: Human-in-the-Loop Machine Learning
Humans in the machine learning life cycleExpert feedback collectionHuman-in-the-loop modelingSummaryQuestionsReferences
Assessments
Chapter 1 – Beyond Code DebuggingChapter 2 – Machine Learning Life CycleChapter 3 – Debugging toward Responsible AIChapter 4 – Detecting Performance and Efficiency Issues in Machine Learning ModelsChapter 5 – Improving the Performance of Machine Learning ModelsChapter 6 – Interpretability and Explainability in Machine Learning ModelingChapter 7 – Decreasing Bias and Achieving FairnessChapter 8 – Controlling Risks Using Test-Driven DevelopmentChapter 9 – Testing and Debugging for ProductionChapter 10 – Versioning and Reproducible Machine Learning ModelingChapter 11 – Avoiding and Detecting Data and Concept DriftsChapter 12 – Going Beyond ML Debugging with Deep LearningChapter 13 – Advanced Deep Learning TechniquesChapter 14 – Introduction to Recent Advancements in Machine LearningChapter 15 – Correlation versus CausalityChapter 16 – Security and Privacy in Machine LearningChapter 17 – Human-in-the-Loop Machine Learning
Index
Why subscribe?
Other Books You May EnjoyPackt is searching for authors like youShare your thoughtsDownload a free PDF copy of this book

Content preview from Debugging Machine Learning Models with Python

4 Detecting Performance and Efficiency Issues in Machine Learning Models

One of the main objectives we must keep in mind is how to build a high-performance machine learning model with minimal errors on new data we want to use the model for. In this chapter, you will learn how to properly assess the performance of your models and identify opportunities for decreasing their errors.

This chapter includes many figures and code examples to help you better understand these concepts and start benefiting from them in your projects.

We will cover the following topics:

Performance and error assessment measures
Visualization
Bias and variance diagnosis
Model validation strategy
Error analysis
Beyond performance

By the end of this chapter, you will have ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Machine Learning Model Serving Patterns and Best Practices

Publisher Resources

ISBN: 9781800208582

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Debugging Machine Learning Models with Python

by Ali Madani

4

Detecting Performance and Efficiency Issues in Machine Learning Models

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.