CHAPTER 9

Markov Processes

In this chapter, we study a class of random processes that possess a certain characteristic that could crudely be described as memoryless. These processes appear in numerous applications including queuing systems, computer communication networks, biological systems, and a wide variety of other applications. As a result of their frequent occurrence, these processes have been studied extensively, and a wealth of theory exists to solve problems related to these processes. We make no attempt to give an exhaustive treatment here, but rather present some of the fundamental concepts involving Markov processes.

9.1 Definition and Examples of Markov Processes

Definition 9.1: A random process, X(t), is said to be a Markov process ...

Get Probability and Random Processes, 2nd Edition now with O’Reilly online learning.

O’Reilly members experience live online training, plus books, videos, and digital content from 200+ publishers.