In order to discuss the MCMC algorithms, it's necessary to introduce the concept of Markov chains. In fact, while the direct sample method draws samples without any particular order, the MCMC strategies draw a sequence of samples according to a precise transition probability from a sample to the following one.
Let's consider a time-dependent random variable X(t), and let's assume a discrete time sequence X1, X2, ..., Xt, Xt+1, ... where Xt represents the value assumed at time t. In the following diagram, there's a schematic representation of this sequence:
Structure of a generic Markov chain
We can suppose ...
