Conformal Prediction for Reliable Machine Learning

Conformal Prediction for Reliable Machine Learning

by Vineeth Balasubramanian, Shen-Shyang Ho, Vladimir Vovk
Released April 2014
Publisher(s): Morgan Kaufmann
ISBN: 9780124017153

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

The conformal predictions framework is a recent development in machine learning that can associate a reliable measure of confidence with a prediction in any real-world pattern recognition application, including risk-sensitive applications such as medical diagnosis, face recognition, and financial risk prediction. Conformal Predictions for Reliable Machine Learning: Theory, Adaptations and Applications captures the basic theory of the framework, demonstrates how to apply it to real-world problems, and presents several adaptations, including active learning, change detection, and anomaly detection. As practitioners and researchers around the world apply and adapt the framework, this edited volume brings together these bodies of work, providing a springboard for further research as well as a handbook for application in real-world problems.
  • Understand the theoretical foundations of this important framework that can provide a reliable measure of confidence with predictions in machine learning
  • Be able to apply this framework to real-world problems in different machine learning settings, including classification, regression, and clustering
  • Learn effective ways of adapting the framework to newer problem settings, such as active learning, model selection, or change detection

Table of contents

  1. Cover image
  2. Title page
  3. Copyright
  4. Copyright Permissions
    1. Chapter 3
    2. Chapter 6
    3. Chapter 9
    4. Chapter 10
    5. Chapter 11
  5. Contributing Authors
  6. Foreword
  7. Preface
    1. Book Organization
    2. Part I: Theory
    3. Part II: Adaptations
    4. Part III: Applications
    5. Companion Website
    6. Contacting Us
    7. Acknowledgments
  8. Part 1: Theory
    1. Chapter 1. The Basic Conformal Prediction Framework
      1. Abstract
      2. Acknowledgments
      3. 1.1 The Basic Setting and Assumptions
      4. 1.2 Set and Confidence Predictors
      5. 1.3 Conformal Prediction
      6. 1.4 Efficiency in the Case of Prediction without Objects
      7. 1.5 Universality of Conformal Predictors
      8. 1.6 Structured Case and Classification
      9. 1.7 Regression
      10. 1.8 Additional Properties of Validity and Efficiency in the Online Framework
    2. Chapter 2. Beyond the Basic Conformal Prediction Framework
      1. Abstract
      2. Acknowledgments
      3. 2.1 Conditional Validity
      4. 2.2 Conditional Conformal Predictors
      5. 2.3 Inductive Conformal Predictors
      6. 2.4 Training Conditional Validity of Inductive Conformal Predictors
      7. 2.5 Classical Tolerance Regions
      8. 2.6 Object Conditional Validity and Efficiency
      9. 2.7 Label Conditional Validity and ROC Curves
      10. 2.8 Venn Predictors
  9. Part 2: Adaptations
    1. Chapter 3. Active Learning
      1. Abstract
      2. Acknowledgments
      3. 3.1 Introduction
      4. 3.2 Background and Related Work
      5. 3.3 Active Learning Using Conformal Prediction
      6. 3.4 Experimental Results
      7. 3.5 Discussion and Conclusions
    2. Chapter 4. Anomaly Detection
      1. Abstract
      2. 4.1 Introduction
      3. 4.2 Background
      4. 4.3 Conformal Prediction for Multiclass Anomaly Detection
      5. 4.4 Conformal Anomaly Detection
      6. 4.5 Inductive Conformal Anomaly Detection
      7. 4.6 Nonconformity Measures for Examples Represented as Sets of Points
      8. 4.7 Sequential Anomaly Detection in Trajectories
      9. 4.8 Conclusions
    3. Chapter 5. Online Change Detection
      1. Abstract
      2. 5.1 Introduction
      3. 5.2 Related Work
      4. 5.3 Background
      5. 5.4 A Martingale Approach for Change Detection
      6. 5.5 Experimental Results
      7. 5.6 Implementation Issues
      8. 5.7 Conclusions
    4. Chapter 6. Feature Selection
      1. Abstract
      2. 6.1 Introduction
      3. 6.2 Feature Selection Methods
      4. 6.3 Issues in Feature Selection
      5. 6.4 Feature Selection for Conformal Predictors
      6. 6.5 Discussion and Conclusions
    5. Chapter 7. Model Selection
      1. Abstract
      2. Acknowledgments
      3. 7.1 Introduction
      4. 7.2 Background
      5. 7.3 SVM Model Selection Using Nonconformity Measure
      6. 7.4 Nonconformity Generalization Error Bound
      7. 7.5 Experimental Results
      8. 7.6 Conclusions
    6. Chapter 8. Prediction Quality Assessment
      1. Abstract
      2. Acknowledgments
      3. 8.1 Introduction
      4. 8.2 Related Work
      5. 8.3 Generalized Transductive Reliability Estimation
      6. 8.4 Experimental Results
      7. 8.5 Discussion and Conclusions
    7. Chapter 9. Other Adaptations
      1. Abstract
      2. Acknowledgments
      3. 9.1 Introduction
      4. 9.2 Metaconformal Predictors
      5. 9.3 Single-Stacking Conformal Predictors
      6. 9.4 Conformal Predictors for Time Series Analysis
      7. 9.5 Conclusions
  10. Part 3: Applications
    1. Chapter 10. Biometrics and Robust Face Recognition
      1. Abstract
      2. 10.1 Introduction
      3. 10.2 Biometrics and Forensics
      4. 10.3 Face Recognition
      5. 10.4 Randomness and Complexity
      6. 10.5 Transduction
      7. 10.6 Nonconformity Measures for Face Recognition
      8. 10.7 Open and Closed Set Face Recognition
      9. 10.8 Watch List and Surveillance
      10. 10.9 Score Normalization
      11. 10.10 Recognition-by-Parts Using Transduction and Boosting
      12. 10.11 Reidentification Using Sensitivity Analysis and Revision
      13. 10.12 Conclusions
    2. Chapter 11. Biomedical Applications: Diagnostic and Prognostic
      1. Abstract
      2. Acknowledgments
      3. 11.1 Introduction
      4. 11.2 Examples of Medical Diagnostics
      5. 11.3 Nonconformity Measures for Medical and Biological Applications
      6. 11.4 Discussion and Conclusions
    3. Chapter 12. Network Traffic Classification and Demand Prediction
      1. Abstract
      2. 12.1 Introduction
      3. 12.2 Network Traffic Classification
      4. 12.3 Network Demand Prediction
      5. 12.4 Experimental Results
      6. 12.5 Conclusions
    4. Chapter 13. Other Applications
      1. Abstract
      2. 13.1 Nuclear Fusion Device Applications
      3. 13.2 Sensor Device Applications
      4. 13.3 Sustainability, Environment, and Civil Engineering
      5. 13.4 Security Applications
      6. 13.5 Applications from Other Domains
  11. Bibliography
  12. Index

Product information

  • Title: Conformal Prediction for Reliable Machine Learning
  • Author(s): Vineeth Balasubramanian, Shen-Shyang Ho, Vladimir Vovk
  • Release date: April 2014
  • Publisher(s): Morgan Kaufmann
  • ISBN: 9780124017153