Bayesian Statistics: An Introduction, 4th Edition

2.9 The role of sufficiency

2.9.1 Definition of sufficiency

When we considered the normal variance with known mean, we found that the posterior distribution depended on the data only through the single number S. It often turns out that the data can be reduced in a similar way to one or two numbers, and as long as we know them we can forget the rest of the data. It is this notion that underlies the formal definition of sufficiency.

Suppose observations are made with a view to gaining knowledge about a parameter θ, and that

is a function of the observations. We call such a function a statistic. We often suppose that t is real valued, but it is sometimes vector valued. Using the formulae in Section 1.4 on ‘Several Random Variables’ and the fact that once we know x we automatically know the value of t, we see that for any statistic t

However, it sometimes happens that

does not depend on θ, so that

If this happens, we say that t is a sufficient statistic for θ given X, often abbreviated ...