book

Machine Learning for High-Risk Applications

by Patrick Hall, James Curtis, Parul Pandey

April 2023

Intermediate to advanced

466 pages

14h 40m

English

O'Reilly Media, Inc.

Read now

Unlock full access

Foreword
Preface
Who Should Read This BookWhat Readers Will LearnAlignment with the NIST AI Risk Management FrameworkBook OutlinePart IPart IIPart IIIExample DatasetsTaiwan Credit DataKaggle Chest X-Ray DataConventions Used in This BookOnline FiguresUsing Code ExamplesO’Reilly Online LearningHow to Contact UsAcknowledgmentsPatrick HallJames CurtisParul Pandey
I. Theories and Practical Applications of AI Risk Management
1. Contemporary Machine Learning Risk Management
A Snapshot of the Legal and Regulatory LandscapeThe Proposed EU AI ActUS Federal Laws and RegulationsState and Municipal LawsBasic Product LiabilityFederal Trade Commission EnforcementAuthoritative Best PracticesAI IncidentsCultural Competencies for Machine Learning Risk ManagementOrganizational AccountabilityCulture of Effective ChallengeDiverse and Experienced TeamsDrinking Our Own ChampagneMoving Fast and Breaking ThingsOrganizational Processes for Machine Learning Risk ManagementForecasting Failure ModesModel Risk Management ProcessesBeyond Model Risk ManagementCase Study: The Rise and Fall of Zillow’s iBuyingFalloutLessons LearnedResources
2. Interpretable and Explainable Machine Learning
Important Ideas for Interpretability and ExplainabilityExplainable ModelsAdditive ModelsDecision TreesAn Ecosystem of Explainable Machine Learning ModelsPost Hoc ExplanationFeature Attribution and ImportanceSurrogate ModelsPlots of Model PerformanceCluster ProfilingStubborn Difficulties of Post Hoc Explanation in PracticePairing Explainable Models and Post Hoc ExplanationCase Study: Graded by AlgorithmResources
3. Debugging Machine Learning Systems for Safety and Performance
TrainingReproducibilityData QualityModel Specification for Real-World OutcomesModel DebuggingSoftware TestingTraditional Model AssessmentCommon Machine Learning BugsResidual AnalysisSensitivity AnalysisBenchmark ModelsRemediation: Fixing BugsDeploymentDomain SafetyModel MonitoringCase Study: Death by Autonomous VehicleFalloutAn Unprepared Legal SystemLessons LearnedResources
4. Managing Bias in Machine Learning
ISO and NIST Definitions for BiasSystemic BiasStatistical BiasHuman Biases and Data Science CultureLegal Notions of ML Bias in the United StatesWho Tends to Experience Bias from ML SystemsHarms That People ExperienceTesting for BiasTesting DataTraditional Approaches: Testing for Equivalent OutcomesA New Mindset: Testing for Equivalent Performance QualityOn the Horizon: Tests for the Broader ML EcosystemSummary Test PlanMitigating BiasTechnical Factors in Mitigating BiasThe Scientific Method and Experimental DesignBias Mitigation ApproachesHuman Factors in Mitigating BiasCase Study: The Bias Bug BountyResources
5. Security for Machine Learning
Security BasicsThe Adversarial MindsetCIA TriadBest Practices for Data ScientistsMachine Learning AttacksIntegrity Attacks: Manipulated Machine Learning OutputsConfidentiality Attacks: Extracted InformationGeneral ML Security ConcernsCountermeasuresModel Debugging for SecurityModel Monitoring for SecurityPrivacy-Enhancing TechnologiesRobust Machine LearningGeneral CountermeasuresCase Study: Real-World Evasion AttacksEvasion AttacksLessons LearnedResources
II. Putting AI Risk Management into Action
6. Explainable Boosting Machines and Explaining XGBoost
Concept Refresher: Machine Learning TransparencyAdditivity Versus InteractionsSteps Toward Causality with ConstraintsPartial Dependence and Individual Conditional ExpectationShapley ValuesModel DocumentationThe GAM Family of Explainable ModelsElastic Net–Penalized GLM with Alpha and Lambda SearchGeneralized Additive ModelsGA2M and Explainable Boosting MachinesXGBoost with Constraints and Post Hoc ExplanationConstrained and Unconstrained XGBoostExplaining Model Behavior with Partial Dependence and ICEDecision Tree Surrogate Models as an Explanation TechniqueShapley Value ExplanationsProblems with Shapley valuesBetter-Informed Model SelectionResources

7. Explaining a PyTorch Image Classifier
Explaining Chest X-Ray ClassificationConcept Refresher: Explainable Models and Post Hoc Explanation TechniquesExplainable Models OverviewOcclusion MethodsGradient-Based MethodsExplainable AI for Model DebuggingExplainable ModelsProtoPNet and VariantsOther Explainable Deep Learning ModelsTraining and Explaining a PyTorch Image ClassifierTraining DataAddressing the Dataset Imbalance ProblemData Augmentation and Image CroppingModel TrainingEvaluation and MetricsGenerating Post Hoc Explanations Using CaptumEvaluating Model ExplanationsThe Robustness of Post Hoc ExplanationsConclusionResources
8. Selecting and Debugging XGBoost Models
Concept Refresher: Debugging MLModel SelectionSensitivity AnalysisResidual AnalysisRemediationSelecting a Better XGBoost ModelSensitivity Analysis for XGBoostStress Testing XGBoostStress Testing MethodologyAltering Data to Simulate Recession ConditionsAdversarial Example SearchResidual Analysis for XGBoostAnalysis and Visualizations of ResidualsSegmented Error AnalysisModeling ResidualsRemediating the Selected ModelOveremphasis of PAY_0Miscellaneous BugsConclusionResources
9. Debugging a PyTorch Image Classifier
Concept Refresher: Debugging Deep LearningDebugging a PyTorch Image ClassifierData Quality and LeaksSoftware Testing for Deep LearningSensitivity Analysis for Deep LearningRemediationSensitivity FixesConclusionResources
10. Testing and Remediating Bias with XGBoost
Concept Refresher: Managing ML BiasModel TrainingEvaluating Models for BiasTesting Approaches for GroupsIndividual FairnessProxy BiasRemediating BiasPreprocessingIn-processingPostprocessingModel SelectionConclusionResources
11. Red-Teaming XGBoost
Concept RefresherCIA TriadAttacksCountermeasuresModel TrainingAttacks for Red-TeamingModel Extraction AttacksAdversarial Example AttacksMembership AttacksData PoisoningBackdoorsConclusionResources
III. Conclusion
12. How to Succeed in High-Risk Machine Learning
Who Is in the Room?Science Versus EngineeringThe Data-Scientific MethodThe Scientific MethodEvaluation of Published Results and ClaimsApply External StandardsCommonsense Risk MitigationConclusionResources
Index
About the Authors

Content preview from Machine Learning for High-Risk Applications

About the Authors

Patrick Hall is principal scientist at BNH.AI, where he advises Fortune 500 companies and cutting-edge startups on AI risk and conducts research in support of NIST’s AI Risk Management Framework. He also serves as visiting faculty in the Department of Decision Sciences at the George Washington School of Business, teaching data ethics, business analytics, and machine learning classes.

Before cofounding BNH, Patrick led H2O.ai’s efforts in responsible AI, resulting in one of the world’s first commercial applications for explainability and bias mitigation in machine learning. He also held global customer-facing roles and R&D research roles at SAS Institute. Patrick studied computational chemistry at the University of Illinois before graduating from the Institute for Advanced Analytics at North Carolina State University.

Patrick has been invited to speak on topics relating to explainable AI at the National Academies of Science, Engineering, and Medicine, ACM SIG-KDD, and the Joint Statistical Meetings. He has contributed written pieces to outlets like McKinsey.com, O’Reilly Radar, and Thompson Reuters Regulatory Intelligence, and his technical work has been profiled in Fortune, Wired, InfoWorld, TechCrunch, and others.

James Curtis is a quantitative researcher at Solea Energy, where he is focused on using statistical forecasting to further the decarbonization of the US power grid. He previously served as a consultant for financial services organizations, insurers, ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Building Machine Learning Powered Applications

Publisher Resources

ISBN: 9781098102425Errata Page

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Machine Learning for High-Risk Applications

by Patrick Hall, James Curtis, Parul Pandey

About the Authors

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.