Markov Chains: Analytic and Monte Carlo Computations

Solutions for Chapter 3

1. 3.1.a The convergences follow from and . The monotone convergence theorem or the Abel theorem allows to conclude.
2. 3.1.b The first result follows from the strong Markov property (Theorem 2.1.3). The second from classic result on products of power series and convolutions.
3. 3.1.c For the first, use
   
4. For the second, use is equal to
   
5. 3.1.d If is odd, then , if is even, then , and
   
6. As with equality if and only if , then for and for . Thus, the random walk is recurrent ...