Appendix C

Pool adjacent violators algorithm

In this appendix we provide a brief description of the PAV algorithm used in Chapter 6. Assume that there exist a validation set of posterior probabilities (Appendix A) that H1 is true, namely Pi, and the respective ground-truth labels of the proposition that is actually true in each case, namely yi {0, 1}, where i is an index representing the case in the validation set for which a posterior probability Pi is assigned. This index i {1, …, N}, where N = N1 + N2, N1 is the number of cases in the validation set for which H1 is true, and N2 is the number of cases in the validation set for which H2 is true. Thus, N is the total number of cases in the validation set. With this notation, when yi = 1 then H1 is true for case i, and when yi = 0 then H2 is true for case i. Therefore, Pi = P(yi = 1). Moreover, without loss of generality, it is assumed that for two cases j and k in the validation set, Pj ≤ Pk if j < k, or in other words, that the probabilities Pi are sorted in ascending order by case index i. Each yi has a value of yi = 1 with probability Pi if H1 is true and yi = 0 with probability Pi if H2 is true. Therefore, the yi values are in fact the oracle probabilities that an oracle forecaster would elicit, as defined in Section 6.5.1. ...

Start Free Trial

No credit card required