For some distributions, such as the Gaussian, we have very efficient algorithms to get samples from, but for other distributions, this is not the case. Metropolis-Hastings enables us to obtain samples from any probability distribution, , given that we can compute at least a value proportional to it, thus ignoring the normalization factor. This is very useful since in a lot of problems, not just Bayesian statistics, the hard part is to compute the normalization factor.
To conceptually understand this method, we are going to use the following analogy. Suppose we are interested in finding the volume of water a lake contains ...