Chapter 7Markov Process

 

 

 

The Markovian modeling of a dynamic system often leads to a Markov chain, for which the sojourn time in each state becomes random. In many cases, this description is insufficient to establish interesting mathematical properties. In that purpose, we introduce the formalism of Markov jump processes, with their semi-groups and infinitesimal generators.

To go one step further and, in particular, to prove several crucial results such as PASTA or its avatars, we need to see a Markov process as the solution of a martingale problem. In this chapter, we review these different characterizations, and show that they are in fact equivalent.

Throughout this chapter, E denotes a state space that is at the most countable, and equipped with the discrete topology. We refer the reader to the definitions and notations of Appendix A.1.

7.1. Preliminaries

We start by stating two technical Lemmas on the exponential distribution, which will be useful in the following.

LEMMA 7.1.– Let U and V be the two independent random variables, of respective distributions ε (λ) and ε (μ), where λ,μ > 0. Then,

images

Proof.

(i) The density of the random couple (U, V) is given for all (u, v) by

images

By denoting the subset A = {(u, v) ∈ R2; uv}, we can write

(ii) It suffices to see that for any

Get Stochastic Modeling and Analysis of Telecoms Networks now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.