Chapter 4
Limit Theorems
4.1 Introduction
In advanced studies of probability theory, limit theorems form the most important class of results. A limit theorem typically starts with a sequence of random variables, X1, X2, . . . and investigates properties of some function of X1, X2, . . ., Xn as n→ ∞. From a practical point of view, this allows us to use the limit as an approximation to an exact quantity that may be difficult to compute.
When we introduced expected values, we argued that these could be considered averages of a large number of observations. Thus, if we have observations X1, X2, . . ., Xn and we do not know the mean μ, a reasonable approximation ought to be the sample mean
in other words, the average of X1, . . ., Xn. Suppose now that the Xk are i.i.d. with mean μ and variance σ2. By the formulas for the mean and variance of sums of independent random variables, we get
and
that is, 
has the same expected value as each individual Xk and a variance that becomes smaller the larger the value of n. This indicates that is likely to be close to μ for large values of n. Since ...
