Robust Bayesian Inference via Coarsening

Jeffrey W. Miller, David B. Dunson

Journal of the American Statistical Association · 2018 · 171 citations · 36 references

DOIFull text

Open access

Concepts

TL;DR

Standard Bayesian inference assumes data come from the chosen model class, but even minor violations can greatly affect results. The paper proposes conditioning on model-generated data close to observations to enhance robustness against small model misspecifications. By defining closeness through relative entropy, the coarsened posterior is approximated by tempering the likelihood, enabling standard algorithms and analytic solutions with conjugate priors. The authors derive theoretical properties and demonstrate the method on real and simulated mixture and autoregressive models. Supplementary materials are available online.

Abstract

The standard approach to Bayesian inference is based on the assumption that the distribution of the data belongs to the chosen model class. However, even a small violation of this assumption can have a large impact on the outcome of a Bayesian procedure. We introduce a novel approach to Bayesian inference that improves robustness to small departures from the model: rather than conditioning on the event that the observed data are generated by the model, one conditions on the event that the model generates data close to the observed data, in a distributional sense. When closeness is defined in terms of relative entropy, the resulting “coarsened” posterior can be approximated by simply tempering the likelihood—that is, by raising the likelihood to a fractional power—thus, inference can usually be implemented via standard algorithms, and one can even obtain analytical solutions when using conjugate priors. Some theoretical properties are derived, and we illustrate the approach with real and simulated data using mixture models and autoregressive models of unknown order. Supplementary materials for this article are available online.

References

36