Concepedia

Understanding the Metropolis-Hastings Algorithm

Siddhartha Chib, Edward Greenberg

The American Statistician · 1995 · 3.7K citations · 24 references

Concepts

TL;DR

The paper provides an introductory exposition of the Metropolis‑Hastings algorithm for simulating multivariate distributions. The authors derive the algorithm intuitively, explain implementation steps, and illustrate two applications—acceptance‑rejection sampling without a blanket function and block‑at‑a‑time scans—with examples. They show that many algorithms, such as the Gibbs sampler, are special cases of Metropolis‑Hastings. Keywords: Gibbs sampling, Markov chain Monte Carlo, multivariate density simulation, reversible Markov chains.

Abstract

Abstract We provide a detailed, introductory exposition of the Metropolis-Hastings algorithm, a powerful Markov chain method to simulate multivariate distributions. A simple, intuitive derivation of this method is given along with guidance on implementation. Also discussed are two applications of the algorithm, one for implementing acceptance-rejection sampling when a blanketing function is not available and the other for implementing the algorithm with block-at-a-time scans. In the latter situation, many different algorithms, including the Gibbs sampler, are shown to be special cases of the Metropolis-Hastings algorithm. The methods are illustrated with examples. Key Words: Gibbs samplingMarkov chain Monte CarloMultivariate density simulationReversible Markov chains

References

24

Equation of State Calculations by Fast Computing Machines

N. Metropolis, Arianna W. Rosenbluth, M. N. Rosenbluth et al. · The Journal of Chemical Physics · 1953

+22

36.5K citations

Time Series Analysis: Forecasting and Control

Michael D. Geurts, George E. P. Box, Gwilym M. Jenkins · Journal of Marketing Research · 1977

+4

19.3K citations

Probability and Measure.

PE, P. Billingsley · Journal of the American Statistical Association · 1996

+14

6.7K citations