Assessing and Improving Prediction and Classification: Theory and Algorithms in C++

© Timothy Masters 2018

Timothy MastersAssessing and Improving Prediction and Classificationhttps://doi.org/10.1007/978-1-4842-3336-8_9

9. Information and Entropy

Timothy Masters¹ 

(1)

Ithaca, New York, USA

 

• Entropy

• Joint and Conditional Entropy

• Mutual Information

• Continuous Mutual Information

• Predictor Selection

An effective model takes one or more predictor variables , processes the information that they contain, and estimates a predicted variable that is ideally useful in some way. But even the most sophisticated model is helpless if it is not given the information it needs to make a good decision. In this chapter, we explore the concept of information content of a variable, and we present a variety of algorithms for assessing the amount and nature ...