Probability, Random Variables, and Random Processes: Theory and Signal Processing Applications

6.10 MARKOV PROCESS

Consider the continuous-time random process X(t) at several time instants t1<···<tN−1<tN such that we can write the joint pdf . In general, there may be some dependence across time instants of the process, or the random variables may be independent, in which case the joint pdf splits into the product of the N marginal pdfs . We are interested in the Markov property for continuous-time random processes.

Definition: Markov Process Random process X(t) is a Markov process if the conditional pdf simplifies as

(6.140)

The Markov property can also be expressed using a conditional cdf:

(6.141)

As in the discrete-time case, the conditioning reduces to include only the random variable at the most recent time instant defined by the conditioning. The history of a particular realization before time tN−1 is not relevant.

The term Markov process is often used to describe any process with the property in the definition above, for continuous or discrete time. If the process has a finite or countable number of states, then it is generally called a Markov chain. The Markov chains ...