A Signal Theoretic Introduction to Random Processes

4BACKGROUND: PROBABILITY AND RANDOM VARIABLE THEORY

4.1 INTRODUCTION

The characterization of a random process requires an understanding of random variable theory, and this has its basis in probability theory. This chapter provides an overview of the key concepts from probability theory that underpin random variable theory. This is followed by an introduction to random variable theory and an overview of discrete and continuous random variables. The concept of expectation of a random variable is fundamental and a brief discussion is included. As part of this discussion, the characteristic function of a random variable is defined. Random variable theory is extended by the generalization to pairs of random variables and a vector of random variables. Associated concepts for this case include the joint probability mass and density functions, marginal distribution and density functions, conditional probability mass and density functions, and covariance and correlation functions. The chapter concludes with a discussion of Stirling’s formula and the important DeMoivre–Laplace and Poisson approximations to the binomial probability mass function. Useful references for probability and random variable theory include Grimmett and Stirzaker (1992) and Loeve (1977).

4.2 BASIC CONCEPTS: EXPERIMENTS-PROBABILITY THEORY

The following definitions are fundamental and define the basic concepts that underpin probability and random process theory.

4.2.1 Experiments and Sample Spaces

Definition: ...