book

Practicing Trustworthy Machine Learning

by Yada Pruksachatkun, Matthew Mcateer, Subho Majumdar

January 2023

Intermediate to advanced

300 pages

8h 8m

English

O'Reilly Media, Inc.

Read now

Unlock full access

Preface
Implementing Machine Learning in ProductionThe Transformer ConvergenceAn Explosion of Large and Highly Capable ML ModelsWhy We Wrote This BookWho This Book Is ForAI Safety and AlignmentUse of HuggingFace PyTorch for AI ModelsFoundationsConventions Used in This BookUsing Code ExamplesO’Reilly Online LearningHow to Contact UsAcknowledgments
1. Privacy
Attack Vectors for Machine Learning PipelinesImproperly Implemented Privacy Features in ML: Case StudiesCase 1: Apple’s CSAMCase 2: GitHub CopilotCase 3: Model and Data Theft from No-Code ML ToolsDefinitionsDefinition of PrivacyProxies and Metrics for PrivacyLegal Definitions of Privacyk-AnonymityTypes of Privacy-Invading Attacks on ML PipelinesMembership AttacksModel InversionModel ExtractionStealing a BERT-Based Language ModelDefenses Against Model Theft from Output LogitsPrivacy-Testing ToolsMethods for Preserving PrivacyDifferential PrivacyStealing a Differentially Privately Trained ModelFurther Differential Privacy ToolingHomomorphic EncryptionSecure Multi-Party ComputationSMPC ExampleFurther SMPC ToolingFederated LearningConclusion
2. Fairness and Bias
Case 1: Social MediaCase 2: Triaging Patients in Healthcare SystemsCase 3: Legal SystemsKey Concepts in Fairness and Fairness-Related HarmsIndividual FairnessParity FairnessCalculating Parity FairnessScenario 1: Language GenerationScenario 2: Image CaptioningFairness Harm MitigationMitigation Methods in the Pre-Processing StageMitigation Methods in the In-Processing StageMitigation Methods in the Post-Processing StageFairness Tool KitsHow Can You Prioritize Fairness in Your Organization?ConclusionFurther Reading
3. Model Explainability and Interpretability
Explainability Versus InterpretabilityThe Need for Interpretable and Explainable ModelsA Possible Trade-off Between Explainability and PrivacyEvaluating the Usefulness of Interpretation or Explanation MethodsDefinitions and Categories“Black Box”Global Versus Local InterpretabilityModel-Agnostic Versus Model-Specific MethodsInterpreting GPT-2Methods for Explaining Models and Interpreting OutputsInherently Explainable ModelsLocal Model-Agnostic Interpretability MethodsGlobal Model-Agnostic Interpretability MethodsExplaining Neural NetworksSaliency MappingDeep Dive: Saliency Mapping with CLIPAdversarial Counterfactual ExamplesOvercome the Limitations of Interpretability with a Security MindsetLimitations and Pitfalls of Explainable and Interpretable MethodsRisks of Deceptive InterpretabilityConclusion
4. Robustness
Evaluating RobustnessNon-Adversarial RobustnessStep 1: Apply PerturbationsStep 2: Defining and Applying ConstraintsDeep Dive: Word Substitution with Cosine Similarity ConstraintsAdversarial RobustnessDeep Dive: Adversarial Attacks in Computer VisionCreating Adversarial ExamplesImproving RobustnessConclusion
5. Secure and Trustworthy Data Generation
Case 1: Unsecured AWS BucketsCase 2: Clearview AI Scraping Photos from Social MediaCase 3: Improperly Stored Medical DataIssues in Procuring Real-World DataUsing the Right Data for the Modeling GoalConsentPII, PHI, and SecretsProportionality and Sampling TechniquesUndescribed VariationUnintended ProxiesFailures of External ValidityData IntegritySetting Reasonable ExpectationsTools for Addressing Data Collection IssuesSynthetically Generated DataDALL·E, GPT-3, and Synthetic DataImproving Pattern Recognition with Synthetic DataDeep Dive: Pre-Training a Model with a Process-Driven Synthetic DatasetFacial Recognition, Pose Detection, and Human-Centric TasksObject Recognition and Related TasksEnvironment NavigationUnity and Unreal EnvironmentsLimitations of Synthetic Data in HealthcareLimitations of Synthetic Data in NLPSelf-Supervised Learned Models Versus Giant Natural DatasetsRepurposing Quality Control Metrics for Security PurposesConclusion
6. More State-of-the-Art Research Questions
Making Sense of Improperly Overhyped Research ClaimsShallow Human-AI Comparison AntipatternDownplaying the Limitations of the Technique AntipatternUncritical PR Piece AntipatternHyperbolic or Just Plain Wrong AntipatternGetting Past These AntipatternsQuantized MLTooling for Quantized MLPrivacy, Bias, Interpretability, and Stability in Quantized MLDiffusion-Based Energy ModelsHomomorphic EncryptionSimulating Federated LearningQuantum Machine LearningTooling and Resources for Quantum Machine LearningWhy QML Will Not Solve Your Regular ML ProblemsMaking the Leap from Theory to Practice
7. From Theory to Practice
Part I: Additional Technical FactorsCausal Machine LearningSparsity and Model CompressionUncertainty QuantificationPart II: Implementation ChallengesMotivating Stakeholders to Develop Trustworthy ML SystemsTrust DebtsImportant Aspects of TrustEvaluation and FeedbackTrustworthiness and MLOpsConclusion
8. An Ecosystem of Trust
ToolingLiFTDatasheetsModel CardsDAG CardsHuman-in-the-Loop StepsOversight GuidelinesStages of AssessmentThe Need for a Cross-Project ApproachMITRE ATLASBenchmarksAI Incident DatabaseBug BountiesDeep Dive: Connecting the DotsDataPre-ProcessingModel TrainingModel InferenceTrust ComponentsConclusion
A. Synthetic Data Generation Tools

B. Other Interpretability and Explainability Tool Kits
Interpretable or Fair Modeling PackagesOther Python Packages for General Explainability
Index
About the Authors

Content preview from Practicing Trustworthy Machine Learning

Chapter 4. Robustness

We know that ML models are not very generalizable—change a few things to the input, and the model breaks. A model’s ability to be resilient to variation in data is called robustness. To put things intuitively, no matter how good your training data is, the model is going to encounter unexpected things in the real world, and robustness is about making sure it’s ready for them.

Tip

At the core of this problem is the fact that you’re almost always training a model to solve for a proxy problem. For example, if you’re trying to make a dog versus cat image classifier, optimizing an image classifier to minimize the error on your limited training set is just a proxy. Your real goal is to distinguish cats and dogs in all possible cases, but the best you can do is this proxy because your computing resources are finite.

Sometimes, optimizing for your proxy won’t bring you very close to your real goal. Sometimes, if you optimize your proxy too much, your overoptimizing will cause you to do worse on your true goal. AI safety researchers have demonstrated that this applies to every single proxy measure ever (whether it’s an AI or human or group of humans doing the optimizing).¹ Your best bet is to know what to look out for and spot early the signs that you’re overoptimizing.

There are two kinds of robustness: train-time robustness and test-time robustness. Train-time robustness focuses on the model’s ability to be resilient to adversarial examples added to the training ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Publisher Resources

ISBN: 9781098120269Errata Page

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Practicing Trustworthy Machine Learning

by Yada Pruksachatkun, Matthew Mcateer, Subho Majumdar

Chapter 4. Robustness

Tip

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.