Introduction to Markov Processes
The focus of this book is on Markov processes and their applications. In this chapter, we define these processes and discuss some of their applications. The chapters that follow are devoted to the different types of Markov processes.
A stochastic process is called a first-order Markov process if for any , the conditional CDF of for given values of depends only on . That is,
This means that, given ...