Practical Fairness

Book description

Fairness is becoming a paramount consideration for data scientists. Mounting evidence indicates that the widespread deployment of machine learning and AI in business and government is reproducing the same biases we're trying to fight in the real world. But what does fairness mean when it comes to code? This practical book covers basic concerns related to data security and privacy to help data and AI professionals use code that's fair and free of bias.

Many realistic best practices are emerging at all steps along the data pipeline today, from data selection and preprocessing to closed model audits. Author Aileen Nielsen guides you through technical, legal, and ethical aspects of making code fair and secure, while highlighting up-to-date academic research and ongoing legal developments related to fairness and algorithms.

  • Identify potential bias and discrimination in data science models
  • Use preventive measures to minimize bias when developing data modeling pipelines
  • Understand what data pipeline components implicate security and privacy concerns
  • Write data processing and modeling code that implements best practices for fairness
  • Recognize the complex interrelationships between fairness, privacy, and data security created by the use of machine learning models
  • Apply normative and legal concepts relevant to evaluating the fairness of machine learning models

Publisher resources

View/Submit Errata

Table of contents

  1. Preface
    1. Goals of This Book
    2. Practical Notes on the Book
    3. Conventions Used in This Book
    4. Using Code Examples
    5. O’Reilly Online Learning
    6. How to Contact Us
    7. Acknowledgments
  2. 1. Fairness, Technology, and the Real World
    1. Fairness in Engineering Is an Old Problem
    2. Our Fairness Problems Now
      1. Community Norms
      2. Equity and Equality
      3. Security
      4. Privacy
    3. Legal Responses to Fairness in Technology
    4. The Assumptions and Approaches in This Book
    5. What If I’m Skeptical of All This Fairness Talk?
      1. Won’t Fairness Slow Down Innovation?
      2. Are There Any Real-World Consequences for Not Developing Fairness-Aware Practices?
    6. What Is Fairness?
    7. Rules to Code By
  3. 2. Understanding Fairness and the Data Science Pipeline
    1. Metrics for Fairness
      1. Measures of Equity
      2. Measures of Privacy
      3. Measures of Security
    2. Connected Concepts
      1. Privacy and Security
      2. Privacy and Equity
      3. Equality and Security
      4. Accuracy and Fairness
    3. Automated Fairness?
    4. Checklist of Points of Entry for Fairness in the Data Science Pipeline
    5. Concluding Remarks
  4. 3. Fair Data
    1. Ensuring Data Integrity
      1. True Measurements
      2. Proportionality and Sampling Technique
    2. Choosing Appropriate Data
      1. Equity
      2. Privacy
      3. Security
    3. Case Study: Choosing the Right Question for a Data Set and the Right Data Set for a Question
    4. Quality Assurance for a Data Set: Identifying Potential Discrimination
    5. A Timeline for Fairness Interventions
    6. Comprehensive Data-Acquisition Checklist
    7. Concluding Remarks
  5. 4. Fairness Pre-Processing
    1. Simple Pre-Processing Methods
    2. Suppression: The Baseline
    3. Massaging the Data Set: Relabeling
    4. AIF360 Pipeline
      1. Loading the Data
      2. Fairness Metrics
    5. The US Census Data Set
    6. Suppression
    7. Reweighting
      1. How It Works
      2. Code Demonstration
    8. Learning Fair Representations
      1. How It Works
      2. Code Demonstration
    9. Optimized Data Transformations
      1. How It Works
      2. Code Demonstration
    10. Fairness Pre-Processing Checklist
    11. Concluding Remarks
  6. 5. Fairness In-Processing
    1. The Basic Idea
    2. The Medical Data Set
    3. Prejudice Remover
      1. How It Works
      2. Code Demonstration
    4. Adversarial Debiasing
      1. How It Works
      2. Code Demonstration
    5. In-Processing Beyond Antidiscrimination
    6. Model Selection
    7. Concluding Remarks
  7. 6. Fairness Post-Processing
    1. Post-Processing Versus Black-Box Auditing
    2. The Data Set
    3. Equality of Opportunity
      1. How It Works
      2. Code Demonstration
    4. Calibration-Preserving Equalized Odds
      1. How It Works
      2. Code Demonstration
    5. Concluding Remarks
  8. 7. Model Auditing for Fairness and Discrimination
    1. The Parameters of an Audit
    2. Scoping: What Should We Audit?
    3. Black-Box Auditing
      1. Running a Model Through Different Counterfactuals
      2. Model of the Model
      3. Auditing Black-Box Models for Indirect Influence
    4. Concluding Remarks
  9. 8. Interpretable Models and Explainability Algorithms
    1. Interpretation Versus Explanation
    2. Interpretable Models
      1. GLRM: How It Works
      2. Code Demonstration
    3. Explainability Methods
      1. SHAP and LIME: The Workhorses for Local Post Hoc Explanations
      2. Data-Driven Explanation
      3. Explainability Metrics
    4. What Interpretation and Explainability Miss
      1. Attacks on Explainable Machine Learning
    5. Interpretation and Explanation Checklist
    6. Concluding Remarks
  10. 9. ML Models and Privacy
    1. Membership Attacks
      1. How It Works
      2. Code Demonstration
    2. Other Privacy Problems and Attacks
    3. Important Privacy Techniques
    4. Concluding Remarks
  11. 10. ML Models and Security
    1. Evasion Attacks
      1. How It Works
      2. Code Demonstration
      3. Defending Against Adversarial Attacks
      4. Some Evasion Attack Packages
      5. Why Do Evasion Attacks Matter to You?
    2. Poisoning Attacks
      1. How They Work
      2. Defenses Against Poisoning Attacks
      3. Some Poisoning Attack Packages
      4. Why Do Poisoning Attacks Matter to You?
    3. Concluding Remarks
  12. 11. Fair Product Design and Deployment
    1. Reasonable Expectations
      1. Expectations of Moving Targets
      2. Clear Communication
    2. Fiduciary Obligations
    3. Respecting Traditional Spheres of Privacy and Private Life
    4. Value Creation
    5. Complex Systems
      1. The Impact of the Product Life Cycle
      2. The Need for Record Keeping
      3. The Need for Experts
    6. Clear Security Promises and Delineated Limitations
      1. Reasonable Expectations of Security
    7. Possibility of Downstream Control and Verification
      1. Verification Systems and Obligations
      2. Product Iteration Timelines
      3. Tracking Downstream Users
    8. Products That Work Better for Privileged People
    9. Dark Patterns
    10. Fair Products Checklist
    11. Concluding Remarks
  13. 12. Laws for Machine Learning
    1. Personal Data
      1. GDPR
      2. California Consumer Privacy Act
      3. Data Broker Laws
    2. Algorithmic Decision Making
      1. GDPR
      2. Proposed US Laws for Algorithms
    3. Security
      1. HIPAA
      2. FTC Guidance on Cybersecurity
      3. Tort Law
    4. Logical Processes
      1. Right to an Explanation
      2. Freedom of Information Laws
      3. Due Process
    5. Some Application-Specific Laws
      1. Biometrics
      2. Local Ordinances on Facial Recognition
      3. Chat Bots
    6. Concluding Remarks
  14. Index

Product information

  • Title: Practical Fairness
  • Author(s): Aileen Nielsen
  • Release date: December 2020
  • Publisher(s): O'Reilly Media, Inc.
  • ISBN: 9781492075738