Probability, Random Variables, and Random Processes: Theory and Signal Processing Applications

9.3 SUFFICIENT STATISTICS

Sufficient statistics are useful for parameter-estimation problems. Consider the pdf with unknown parameter θ, where the notation emphasizes that θ is the parameter to be estimated. For example, θ might be the mean or variance of an iid Gaussian random sequence with parameters . Suppose we have N samples X and let T be a function (mapping) of the samples. Although the original N observations are condensed to sufficient statistic T, it contains the same information in terms of estimating θ: it is sufficient for estimation purposes. Thus, if T is available and is sufficient, then knowing the original set of observations X provides no additional information for estimating θ.

Definition: Sufficient Statistic Statistic T is sufficient for estimating parameter θ if the conditional probability P(X = x|T = t) is not a function of θ. When the random variables are continuous, T is sufficient for θ if the conditional pdf fX|T(x|t) is not a function of θ.

The conditional probability can be rewritten as

(9.5)

The second expression follows because {x, t} is equivalent to X: we can determine t from X. The numerator and denominator in the last expression both depend on θ, ...