2.9 The role of sufficiency
2.9.1 Definition of sufficiency
When we considered the normal variance with known mean, we found that the posterior distribution depended on the data only through the single number S. It often turns out that the data can be reduced in a similar way to one or two numbers, and as long as we know them we can forget the rest of the data. It is this notion that underlies the formal definition of sufficiency.
Suppose observations are made with a view to gaining knowledge about a parameter θ, and that
is a function of the observations. We call such a function a statistic. We often suppose that t is real valued, but it is sometimes vector valued. Using the formulae in Section 1.4 on ‘Several Random Variables’ and the fact that once we know x we automatically know the value of t, we see that for any statistic t
However, it sometimes happens that
does not depend on θ, so that
If this happens, we say that t is a sufficient statistic for θ given X, often abbreviated ...