Probability, Random Variables, and Random Processes: Theory and Signal Processing Applications

9.5 EXPONENTIAL FAMILIES

In this section, we consider exponential families of random variables for which a sufficient statistic of low dimension is easily found.

Definition: Exponential Family An exponential family of random variables with scalar parameter θ has a pdf that can be expressed in the following form:

(9.42)

where , , c(x), and d(x) are known functions. It is called a family of random variables because the form above remains the same as θ is varied. For N iid random variables {Xn}, (9.42) becomes

(9.43)

For a vector of M parameters and a single random variable, the pdf is

(9.44)

and for N iid random variables and M parameters, (9.44) becomes

(9.45)

An exponential family of random variables should not be confused with the exponential distribution described in Chapter 3 (which ...