Solutions for the exercises
Solutions for Chapter 1
- 1.1 This constitutes a Markov chain on with matrix
- from which the graph is readily deduced. The astronaut can reach any module from any module in a finite number of steps, and hence, the chain is irreducible, and as the state space is finite, this yields that there exists a unique invariant measure . Moreover, and by uniqueness and symmetry, , and hence, . By normalization, we conclude that and .
- 1.2 This constitutes a Markov chain on with matrix
- from which the graph is readily deduced. The mouse can reach one room from any other room ...
Get Markov Chains: Analytic and Monte Carlo Computations now with O’Reilly online learning.
O’Reilly members experience live online training, plus books, videos, and digital content from 200+ publishers.