Chapter 4

Limit Theorems

# 4.1 Introduction

In advanced studies of probability theory, limit theorems form the most important class of results. A limit theorem typically starts with a sequence of random variables, X_{1}, X_{2}, . . . and investigates properties of some function of X_{1}, X_{2}, . . ., X_{n} as n→ ∞. From a practical point of view, this allows us to use the limit as an approximation to an exact quantity that may be difficult to compute.

When we introduced expected values, we argued that these could be considered averages of a large number of observations. Thus, if we have observations X_{1}, X_{2}, . . ., X_{n} and we do not know the mean μ, a reasonable approximation ought to be the sample mean

in other words, the average of X_{1}, . . ., X_{n}. Suppose now that the X_{k} are i.i.d. with mean μ and variance σ^{2}. By the formulas for the mean and variance of sums of independent random variables, we get

and

that is, has the same expected value as each individual X_{k} and a variance that becomes smaller the larger the value of n. This indicates that is likely to be close to μ for large values of n. Since ...