A Variational Baysian Framework for Graphical Models

TLDR

The paper introduces a practical framework for Bayesian model averaging and model selection in probabilistic graphical models. It analytically approximates full posterior distributions over parameters, structures, and latent variables through a free‑form optimization that incorporates conjugate priors and avoids Hessian calculations. The resulting algorithm produces analytic predictive quantities, generalizes EM with guaranteed convergence, and is applicable to mixture models, source separation, and other domains.

Abstract

This paper presents a novel practical framework for Bayesian model averaging and model selection in probabilistic graphical models. Our approach approximates full posterior distributions over model parameters and structures, as well as latent variables, in an analytical manner. These posteriors fall out of a free-form optimization procedure, which naturally incorporates conjugate priors. Unlike in large sample approximations, the posteriors are generally non-Gaussian and no Hessian needs to be computed. Predictive quantities are obtained analytically. The resulting algorithm generalizes the standard Expectation Maximization algorithm, and its convergence is guaranteed. We demonstrate that this approach can be applied to a large class of models in several domains, including mixture models and source separation.