Probability, Random Variables, Statistics, and Random Processes

1Basic Concepts of Probability Theory

Randomness and uncertainty, which always go hand in hand, exist in virtually every aspect of life. To this effect, almost everyone has a basic understanding of the term probability through intuition or experience. The study of probability stems from the analysis of certain games of chance. Probability is the measure of chance that an event will occur, and as such finds applications in disciplines that involve uncertainty. Probability theory is extensively used in a host of areas in science, engineering, medicine, and business, to name just a few. As claimed by Pierre‐Simon Laplace, a prominent French scholar, probability theory is nothing but common sense reduced to calculation. The basic concepts of probability theory are discussed in this chapter.

1.1 Statistical Regularity and Relative Frequency

An experiment is a measurement procedure or observation process. The outcome is the end result of an experiment, where if one outcome occurs, then no other outcome can occur at the same time. An event is a single outcome or a collection of outcomes of an experiment.

If the outcome of an experiment is certain, that is the outcome is always the same, it is then a deterministic experiment. In other words, a deterministic experiment always produces the same output from a given starting condition or initial state. The measurement of the temperature in a certain location at a given time using a thermometer is an example of a deterministic experiment. ...