R Statistics Cookbook

Doing MCMC the manual way

In this recipe, we will go through a full example of coding an MCMC algorithm ourselves. This will give us a much better grasp of MCMC mechanics.

In the Bayesian world, we put prior densities and use data to augment those priors, and get posterior densities. The problem is that there are only a few occasions where we can calculate those posterior densities analytically—these are called conjugate families.

The Bayesian problem can be formulated as recovering the conditional density of the parameter given data. This is equal to the ratio of the joint density of the parameters and the data divided by the marginal density of the data. This follows from Bayes theorem, which states that we can invert a conditional probability ...