Chapter 7
Markov Chain Monte Carlo
When a target density f can be evaluated but not easily sampled, the methods from Chapter 6 can be applied to obtain an approximate or exact sample. The primary use of such a sample is to estimate the expectation of a function of X ~ f(x). The Markov chain Monte Carlo (MCMC) methods introduced in this chapter can also be used to generate a draw from a distribution that approximates f, but they are more properly viewed as methods for generating a sample from which expectations of functions of X can reliably be estimated. MCMC methods are distinguished from the simulation techniques in Chapter 6 by their iterative nature and the ease with which they can be customized to very diverse and difficult problems. Viewed as an integration method, MCMC has several advantages over the approaches in Chapter 5: Increasing problem dimensionality usually does not slow convergence or make implementation more complex.
A quick review of discrete-state-space Markov chain theory is provided in Section 1.7. Let the sequence 
denote a Markov chain for t = 0, 1, 2, . . ., where 
and the state space is either continuous or discrete. For the types of Markov chains introduced in this chapter, the distribution of X(t) converges to the limiting stationary distribution of the ...
