APPENDIX BPROBABILITY AND RANDOM VARIABLES
This Appendix provides a short summary of probability, random variables, and stochastic processes as applied to estimation. Further information may be found in most books on estimation, probability or signal processing; for example, Papoulis and Pillai (2002), Parzen (1960), Fisz (1963), Jazwinski (1970), Levine (1996, Chapter 34), Åström (1970), Whalen (1971), Davenport and Root (1958), Bar-Shalom et al. (2001), Simon (2006), Maybeck (1979), and Anderson and Moore (1979).
B.1 PROBABILITY
B.1.1 Definitions
Probability theory was originally developed to analyze the unpredictability of gambling. More generally, probability is used to describe the average relative occurrence of random events. Events are considered random if the outcome cannot be precisely predicted. For example, the throw of a die results in six possible outcomes, and the outcome can only be predicted as an average fraction of the total number of throws.
The event set or sample space Ω contains samples representing all possible outcomes of an “experiment.” The exposed face of the die defines the outcome of a die toss, and we use the number of dots to represent the face; for example, Ω = {1,2,3,4,5,6}. Since the outcome must be one of these six possibilities, the sample space Ω is a certain event, and the probability of a certain event is defined as one. More generally the sample space is represented as Ω = {A1,A2,…,An} where Ai are the sample outcomes. Certain (measurable) ...
