book

Machine Learning for High-Risk Applications

by Patrick Hall, James Curtis, Parul Pandey

April 2023

Intermediate to advanced

466 pages

14h 40m

English

O'Reilly Media, Inc.

Read now

Unlock full access

Foreword
Preface
Who Should Read This BookWhat Readers Will LearnAlignment with the NIST AI Risk Management FrameworkBook OutlinePart IPart IIPart IIIExample DatasetsTaiwan Credit DataKaggle Chest X-Ray DataConventions Used in This BookOnline FiguresUsing Code ExamplesO’Reilly Online LearningHow to Contact UsAcknowledgmentsPatrick HallJames CurtisParul Pandey
I. Theories and Practical Applications of AI Risk Management
1. Contemporary Machine Learning Risk Management
A Snapshot of the Legal and Regulatory LandscapeThe Proposed EU AI ActUS Federal Laws and RegulationsState and Municipal LawsBasic Product LiabilityFederal Trade Commission EnforcementAuthoritative Best PracticesAI IncidentsCultural Competencies for Machine Learning Risk ManagementOrganizational AccountabilityCulture of Effective ChallengeDiverse and Experienced TeamsDrinking Our Own ChampagneMoving Fast and Breaking ThingsOrganizational Processes for Machine Learning Risk ManagementForecasting Failure ModesModel Risk Management ProcessesBeyond Model Risk ManagementCase Study: The Rise and Fall of Zillow’s iBuyingFalloutLessons LearnedResources
2. Interpretable and Explainable Machine Learning
Important Ideas for Interpretability and ExplainabilityExplainable ModelsAdditive ModelsDecision TreesAn Ecosystem of Explainable Machine Learning ModelsPost Hoc ExplanationFeature Attribution and ImportanceSurrogate ModelsPlots of Model PerformanceCluster ProfilingStubborn Difficulties of Post Hoc Explanation in PracticePairing Explainable Models and Post Hoc ExplanationCase Study: Graded by AlgorithmResources
3. Debugging Machine Learning Systems for Safety and Performance
TrainingReproducibilityData QualityModel Specification for Real-World OutcomesModel DebuggingSoftware TestingTraditional Model AssessmentCommon Machine Learning BugsResidual AnalysisSensitivity AnalysisBenchmark ModelsRemediation: Fixing BugsDeploymentDomain SafetyModel MonitoringCase Study: Death by Autonomous VehicleFalloutAn Unprepared Legal SystemLessons LearnedResources
4. Managing Bias in Machine Learning
ISO and NIST Definitions for BiasSystemic BiasStatistical BiasHuman Biases and Data Science CultureLegal Notions of ML Bias in the United StatesWho Tends to Experience Bias from ML SystemsHarms That People ExperienceTesting for BiasTesting DataTraditional Approaches: Testing for Equivalent OutcomesA New Mindset: Testing for Equivalent Performance QualityOn the Horizon: Tests for the Broader ML EcosystemSummary Test PlanMitigating BiasTechnical Factors in Mitigating BiasThe Scientific Method and Experimental DesignBias Mitigation ApproachesHuman Factors in Mitigating BiasCase Study: The Bias Bug BountyResources
5. Security for Machine Learning
Security BasicsThe Adversarial MindsetCIA TriadBest Practices for Data ScientistsMachine Learning AttacksIntegrity Attacks: Manipulated Machine Learning OutputsConfidentiality Attacks: Extracted InformationGeneral ML Security ConcernsCountermeasuresModel Debugging for SecurityModel Monitoring for SecurityPrivacy-Enhancing TechnologiesRobust Machine LearningGeneral CountermeasuresCase Study: Real-World Evasion AttacksEvasion AttacksLessons LearnedResources
II. Putting AI Risk Management into Action
6. Explainable Boosting Machines and Explaining XGBoost
Concept Refresher: Machine Learning TransparencyAdditivity Versus InteractionsSteps Toward Causality with ConstraintsPartial Dependence and Individual Conditional ExpectationShapley ValuesModel DocumentationThe GAM Family of Explainable ModelsElastic Net–Penalized GLM with Alpha and Lambda SearchGeneralized Additive ModelsGA2M and Explainable Boosting MachinesXGBoost with Constraints and Post Hoc ExplanationConstrained and Unconstrained XGBoostExplaining Model Behavior with Partial Dependence and ICEDecision Tree Surrogate Models as an Explanation TechniqueShapley Value ExplanationsProblems with Shapley valuesBetter-Informed Model SelectionResources

7. Explaining a PyTorch Image Classifier
Explaining Chest X-Ray ClassificationConcept Refresher: Explainable Models and Post Hoc Explanation TechniquesExplainable Models OverviewOcclusion MethodsGradient-Based MethodsExplainable AI for Model DebuggingExplainable ModelsProtoPNet and VariantsOther Explainable Deep Learning ModelsTraining and Explaining a PyTorch Image ClassifierTraining DataAddressing the Dataset Imbalance ProblemData Augmentation and Image CroppingModel TrainingEvaluation and MetricsGenerating Post Hoc Explanations Using CaptumEvaluating Model ExplanationsThe Robustness of Post Hoc ExplanationsConclusionResources
8. Selecting and Debugging XGBoost Models
Concept Refresher: Debugging MLModel SelectionSensitivity AnalysisResidual AnalysisRemediationSelecting a Better XGBoost ModelSensitivity Analysis for XGBoostStress Testing XGBoostStress Testing MethodologyAltering Data to Simulate Recession ConditionsAdversarial Example SearchResidual Analysis for XGBoostAnalysis and Visualizations of ResidualsSegmented Error AnalysisModeling ResidualsRemediating the Selected ModelOveremphasis of PAY_0Miscellaneous BugsConclusionResources
9. Debugging a PyTorch Image Classifier
Concept Refresher: Debugging Deep LearningDebugging a PyTorch Image ClassifierData Quality and LeaksSoftware Testing for Deep LearningSensitivity Analysis for Deep LearningRemediationSensitivity FixesConclusionResources
10. Testing and Remediating Bias with XGBoost
Concept Refresher: Managing ML BiasModel TrainingEvaluating Models for BiasTesting Approaches for GroupsIndividual FairnessProxy BiasRemediating BiasPreprocessingIn-processingPostprocessingModel SelectionConclusionResources
11. Red-Teaming XGBoost
Concept RefresherCIA TriadAttacksCountermeasuresModel TrainingAttacks for Red-TeamingModel Extraction AttacksAdversarial Example AttacksMembership AttacksData PoisoningBackdoorsConclusionResources
III. Conclusion
12. How to Succeed in High-Risk Machine Learning
Who Is in the Room?Science Versus EngineeringThe Data-Scientific MethodThe Scientific MethodEvaluation of Published Results and ClaimsApply External StandardsCommonsense Risk MitigationConclusionResources
Index
About the Authors

Overview

The past decade has witnessed the broad adoption of artificial intelligence and machine learning (AI/ML) technologies. However, a lack of oversight in their widespread implementation has resulted in some incidents and harmful outcomes that could have been avoided with proper risk management. Before we can realize AI/ML's true benefit, practitioners must understand how to mitigate its risks.

This book describes approaches to responsible AI—a holistic framework for improving AI/ML technology, business processes, and cultural competencies that builds on best practices in risk management, cybersecurity, data privacy, and applied social science. Authors Patrick Hall, James Curtis, and Parul Pandey created this guide for data scientists who want to improve real-world AI/ML system outcomes for organizations, consumers, and the public.

Learn technical approaches for responsible AI across explainability, model validation and debugging, bias management, data privacy, and ML security
Learn how to create a successful and impactful AI risk management practice
Get a basic guide to existing standards, laws, and assessments for adopting AI technologies, including the new NIST AI Risk Management Framework
Engage with interactive resources on GitHub and Colab

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Publisher Resources

ISBN: 9781098102425Errata Page

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills