Probability and Stochastic Processes

12.4 Characterization of States for a Markov Chain. Stationary Distribution.

Before we give the main theorem of this chapter, we need to introduce several new concepts.

Definition 12.20

A Markov chain is called irreducible if it only contains one class (in other words, all states communicate with each other).

A Markov chain is called aperiodic if all states have period .

A reducible chain may be split into two or more classes.

Definition 12.21

A probability density is called stationary for the Markov chain if and only if

or equivalently, , where denotes the transposition operation. Thus, if you like the linear algebra language better, is an eigenvector of corresponding ...