Probability, Random Variables, and Random Processes: Theory and Signal Processing Applications

2.1 INTRODUCTION

In this chapter, we present a theory of probability that is based on three axioms and is consistent with our intuition about the probability of an event. This will require a review of some background material on set theory, which will lead to definitions of an event and a field that are needed so that probabilities can be assigned and computed. Perhaps the simplest and most widely used example of a random experiment is the toss of a single coin. Although relatively trivial, we use this experiment often to give definitions and describe properties of random events. Since there are only two outcomes in this experiment, heads (H) and tails (T), it is also a model for a digital communication system which employs a binary alphabet {0, 1} (outcomes {−1, 1} are also used when symmetry about zero is desired). Even though initially the coin-toss experiment appears to be lacking much structure, it is actually quite useful and relevant for many applications.

Another experiment that will be used to demonstrate various properties of probability is the toss of a single die, which has more complexity than the coin-toss experiment because there are multiple outcomes: {1, 2, 3, 4, 5, 6}. Although any experiment with several outcomes would be sufficient for our purposes, we use the die-toss experiment because it is familiar to most readers. This example also has relevance in communications, as a model for pulse amplitude modulation (PAM) where symbols are selected from a finite alphabet ...