Digital Communication Systems

CHAPTER

3

Probability Theory and Bayesian Inference

3.1   Introduction

The idea of a mathematical model used to describe a physical phenomenon is well established in the physical sciences and engineering. In this context, we may distinguish two classes of mathematical models: deterministic and probabilistic. A model is said to be deterministic if there is no uncertainty about its time-dependent behavior at any instant of time; linear time-invariant systems considered in Chapter 2 are examples of a deterministic model. However, in many real-world problems, the use of a deterministic model is inappropriate because the underlying physical phenomenon involves too many unknown factors. In such situations, we resort to a probabilistic model that accounts for uncertainty in mathematical terms.

Probabilistic models are needed for the design of systems that are reliable in performance in the face of uncertainty, efficient in computational terms, and cost effective in building them. Consider for example, a digital communication system that is required to provide practically error-free communication across a wireless channel. Unfortunately, the wireless channel is subject to uncertainties, the sources of which include:

• noise, internally generated due to thermal agitation of electrons in the conductors and electronic devices at the front-end of the receiver;
• fading of the channel, due to the multipath phenomenon—an inherent characteristic of wireless channels;
• interference, representing ...