Chapter 13Semi-Markov Processes and Continuous time Markov Processes

In many situations, we wish to model the time between transitions as well as the nature of the transitions between states. Often, we have a pretty good idea about the distribution of the time spent in a particular state. The semi-Markov process is an example of a process that tries to capture both time and state transitions. However, this process is not Markov in general, in the sense that next transitions may depend on the previous states besides the current one. In a special case where the transitions only depend on the current states, we obtain the Markov process as a particular case of the process presented in this chapter. We decided to include the semi-Markov process in this book since the mathematical treatment of the two processes is very similar. Naturally, since the Markov process is a particular case, it will have extra properties, but the mathematical framework is identical.

Get Probability and Stochastic Processes now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.