Understanding the Metropolis-Hastings Algorithm

TLDR

The paper provides an introductory exposition of the Metropolis‑Hastings algorithm for simulating multivariate distributions. The authors derive the algorithm intuitively, explain implementation steps, and illustrate two applications—acceptance‑rejection sampling without a blanket function and block‑at‑a‑time scans—with examples. They show that many algorithms, such as the Gibbs sampler, are special cases of Metropolis‑Hastings. Keywords: Gibbs sampling, Markov chain Monte Carlo, multivariate density simulation, reversible Markov chains.

Abstract

Abstract We provide a detailed, introductory exposition of the Metropolis-Hastings algorithm, a powerful Markov chain method to simulate multivariate distributions. A simple, intuitive derivation of this method is given along with guidance on implementation. Also discussed are two applications of the algorithm, one for implementing acceptance-rejection sampling when a blanketing function is not available and the other for implementing the algorithm with block-at-a-time scans. In the latter situation, many different algorithms, including the Gibbs sampler, are shown to be special cases of the Metropolis-Hastings algorithm. The methods are illustrated with examples. Key Words: Gibbs samplingMarkov chain Monte CarloMultivariate density simulationReversible Markov chains