R Statistics Cookbook

Bayesian variable selection

Bayesian variable selection within a classical context is usually simple. It really boils down to selecting an appropriate metric (such as the AIC or p-values) and evaluating the model in a greedy way; starting with either a simple (or complex) model, and seeing what happens when we add (or remove) terms.

In a Bayesian context, things are not that easy, since we are not treating parameters as fixed values. We are estimating a posterior density, but a density itself has no significance so we can no longer remove them based on p-values. The AIC way can't be used either, as we don't have an AIC value, but a distribution of possible AICs.

Clearly, we need a different way of doing variable selection that takes into ...