Bayesian Statistics: An Introduction, 4th Edition

2.10 Conjugate prior distributions

2.10.1 Definition and difficulties

When the normal variance was first mentioned, it was stated said that it helps if the prior is of such that the posterior is of a ‘nice’ form, and this led to the suggestion that if a reasonable approximation to your prior beliefs could be managed by using (a multiple of) an inverse chi-squared distribution, it would be sensible to employ this distribution. It is this thought which leads to the notion of conjugate families. The usual definition adopted is as follows:

Let l be a likelihood function . A class Π of prior distributions is said to form a conjugate family if the posterior density

is in the class Π for all X whenever the prior density is in Π.

There is actually a difficulty with this definition, as was pointed out by Diaconis and Ylvisaker (1979 and 1985). If Π is a conjugate family and is any fixed function, then the family of densities proportional to for is also a conjugate family. While this is a logical ...