Solutions for the exercises

Solutions for Chapter 1

  1. 1.1 This constitutes a Markov chain on b03-math-0001 with matrix
    equation
  2. from which the graph is readily deduced. The astronaut can reach any module from any module in a finite number of steps, and hence, the chain is irreducible, and as the state space is finite, this yields that there exists a unique invariant measure b03-math-0003. Moreover, b03-math-0004 and by uniqueness and symmetry, b03-math-0005, and hence, b03-math-0006. By normalization, we conclude that b03-math-0007 and b03-math-0008.
  3. 1.2 This constitutes a Markov chain on b03-math-0009 with matrix
  4. from which the graph is readily deduced. The mouse can reach one room from any other room ...

Get Markov Chains: Analytic and Monte Carlo Computations now with O’Reilly online learning.

O’Reilly members experience live online training, plus books, videos, and digital content from 200+ publishers.