Conformal Prediction for Reliable Machine Learning

Chapter 8

Prediction Quality Assessment

Matjaž Kukar,    Faculty of Computer and Information Science, University of Ljubljana, Ljubljana, Slovenia

Abstract

In the last decade machine learning and data mining were established as highly valuable data analysis tools. Their pervasive use means that they are used in several risk-sensitive domains, where failed predictions may cause substantial financial, economic, health, reputational, or other damage. For such use, most data mining approaches are less than ideal, since more often than not, they cannot produce reliable and unbiased assessments of their predictions’ quality. In recent years, several approaches for estimating reliability or confidence of individual classifiers have emerged, many of them ...