Markov Chains for Exploring Posterior Distributions

TLDR

Markov chain methods such as Gibbs sampling, the Metropolis algorithm, and hybrid strategies are available for sampling from posterior distributions. This paper outlines basic methods and strategies and discusses related theoretical and practical issues. The authors employ general state‑space Markov chain theory to derive convergence rates, laws of large numbers, and central limit theorems, and apply standard simulation techniques for variance reduction and guidance on sample size and allocation. These theoretical results can guide the construction of more efficient algorithms.

Abstract

Several Markov chain methods are available for sampling from a posterior distribution. Two important examples are the Gibbs sampler and the Metropolis algorithm. In addition, several strategies are available for constructing hybrid algorithms. This paper outlines some of the basic methods and strategies and discusses some related theoretical and practical issues. On the theoretical side, results from the theory of general state space Markov chains can be used to obtain convergence rates, laws of large numbers and central limit theorems for estimates obtained from Markov chain methods. These theoretical results can be used to guide the construction of more efficient algorithms. For the practical use of Markov chain methods, standard simulation methodology provides several variance reduction techniques and also give guidance on the choice of sample size and allocation.