Chapter 4

Limit Theorems

4.1 Introduction

In advanced studies of probability theory, limit theorems form the most important class of results. A limit theorem typically starts with a sequence of random variables, X1, X2, . . . and investigates properties of some function of X1, X2, . . ., Xn as n→ ∞. From a practical point of view, this allows us to use the limit as an approximation to an exact quantity that may be difficult to compute.

When we introduced expected values, we argued that these could be considered averages of a large number of observations. Thus, if we have observations X1, X2, . . ., Xn and we do not know the mean μ, a reasonable approximation ought to be the sample mean

img

in other words, the average of X1, . . ., Xn. Suppose now that the Xk are i.i.d. with mean μ and variance σ2. By the formulas for the mean and variance of sums of independent random variables, we get

img

and

img

that is, img has the same expected value as each individual Xk and a variance that becomes smaller the larger the value of n. This indicates that is likely to be close to μ for large values of n. Since ...

Get Probability, Statistics, and Stochastic Processes, 2nd Edition now with the O’Reilly learning platform.

O’Reilly members experience live online training, plus books, videos, and digital content from nearly 200 publishers.