Chapter 8Continuous-Time Markov Chains

8.1 Introduction

The analysis of continuous-time Markov chains (CTMCs) is similar to that of the discrete-time case, except that the transitions from a given state to another state can take place at any instant of time. As in the last chapter, we confine our attention to discrete-state processes. This implies that, although the parameter t has a continuous range of values, the set of values of c08-math-0002 is discrete. Let c08-math-0003 denote the state space of the process, and c08-math-0004 be its parameter space. Recalling from Chapter 6, a discrete-state continuous-time stochastic process c08-math-0005 is called a Markov chain if for c08-math-0006, with t and c08-math-0008, its conditional pmf satisfies the relation

The behavior of the process is characterized by (1) the initial state probability vector of the CTMC ...

Get Probability and Statistics with Reliability, Queuing, and Computer Science Applications, 2nd Edition now with O’Reilly online learning.

O’Reilly members experience live online training, plus books, videos, and digital content from 200+ publishers.