Introduction to Stochastic Models

Chapter 5

Semi-Markov Models

5.1. Introduction

After almost 50 years of research on Markovian dependence—research rich in theoretical and applied results, at the International congress of mathematics held in Amsterdam in 1954, P. Lévy and W. L. Smith introduced a new class of stochastic processes, called by both authors semi-Markov processes. It seems that it was K. L. Chung who had suggested the idea to P. Lévy. Also in 1954, Takács introduced the same type of stochastic process and used it to counter theory applied to the registering of elementary particles.

Starting from that moment, the theory of semi-Markov processes has known an important development, with various fields of application. Nowadays, there is a large literature on this topic ([BAL 79, ÇIN 69b, ÇIN 69a, GIH 74, HAR 76, HOW 64, KOR 82, KOR 93, LIM 01, PYK 61a, PYK 61b, SIL 80, BAR 08], etc.) and, as far as we are concerned, we only want to provide here a general presentation of semi-Markov processes.

Semi-Markov processes are a natural generalization of Markov processes. Markov processes and renewal processes are particular cases of semi-Markov processes. For such a process, its future evolution depends only on the time elapsed from its last transition, and the sojourn time in any state depends only on the present state and on the next state to be visited. As opposed to Markov processes where the sojourn time in a state is exponentially distributed, the sojourn time distribution of a semi-Markov process can be ...