Markov Chains: Analytic and Monte Carlo Computations

Chapter 4Long-time behavior

4.1 Path regeneration and convergence

In this section, let be an irreducible recurrent Markov chain on . For , let the counting measure with integer values, and the empirical measure , which is a probability measure, be the random measures given by

so that if is a real function on , then

4.1.1

and if is in , then

For any state , Lemma 3.1.3 yields that , and hence ...