Algebraic and Stochastic Coding Theory

14

Belief Propagation

14.1  Rational Belief

Bayes’ theorem (Bayes [1764]) is a mathematical statement based on the subjectivists’ notion of belief governed by probability, and it is used to calculate conditional probabilities. In simplest terms, this theorem states that a hypothesis is confirmed by any body of data that its truth renders probable. Thus, the probability of a hypothesis h conditional on some given data d is defined as the ratio of the unconditional probability of the conjunction of the hypothesis with the data to the unconditional probability of the data alone. In other words,

${\text{p}}_d(h)=\frac{\text{p}(h\wedge d)}{\text{p}(d)},$

(14.1.1)

provided that both terms of this ratio exist and p(d) > 0, where p_d is a probability function (Birnbaum [1962]), and ∧ denotes ...