In this chapter, we study a class of random processes that possess a certain characteristic that could crudely be described as memoryless. These processes appear in numerous applications including queuing systems, computer communication networks, biological systems, and a wide variety of other applications. As a result of their frequent occurrence, these processes have been studied extensively, and a wealth of theory exists to solve problems related to these processes. We make no attempt to give an exhaustive treatment here, but rather present some of the fundamental concepts involving Markov processes.
9.1 Definition and Examples of Markov Processes
Definition 9.1: A random process, X(t), is said to be a Markov process ...