A gentle introduction to Markov chains

In order to discuss the MCMC algorithms, it's necessary to introduce the concept of Markov chains. In fact, while the direct sample method draws samples without any particular order, the MCMC strategies draw a sequence of samples according to a precise transition probability from a sample to the following one.

Let's consider a time-dependent random variable X(t), and let's assume a discrete time sequence X1, X2, ..., Xt, Xt+1, ... where Xt represents the value assumed at time t. In the following diagram, there's a schematic representation of this sequence:

 Structure of a generic Markov chain

We can suppose ...

Get Python: Advanced Guide to Artificial Intelligence now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.