Journal of Machine Learning Research · 2013 · 1.5K citations · 71 references
Natural SciencesComputer ScienceMathematicsProbability TheoryStatistical ModelingStatisticsBayesian InferenceBayesian ModelingBayesian StatisticsStatistical InferenceTopic ModelingVariational InferenceBayesian Nonparametric Topic ModelsDirichlet Process ModelsHierarchical Dirichlet ProcessesHierarchical Topic ModelsNonparametric Topic ModelsScalable Topic ModelingStochastic Variational InferenceVariational Posterior Approximation
The paper introduces stochastic variational inference, a scalable algorithm for approximating posterior distributions. The method applies to a broad class of probabilistic models, demonstrated on latent Dirichlet allocation and hierarchical Dirichlet process topic models, and is used to analyze large document collections from Nature, The New York Times, and Wikipedia. Stochastic variational inference scales to millions of documents, outperforming traditional variational inference and enabling Bayesian nonparametric topic models to surpass parametric counterparts on massive datasets.
We develop stochastic variational inference, a scalable algorithm for approximating posterior distributions. We develop this technique for a large class of probabilistic models and we demonstrate it with two probabilistic topic models, latent Dirichlet allocation and the hierarchical Dirichlet process topic model. Using stochastic variational inference, we analyze several large collections of documents: 300K articles from Nature, 1.8M articles from The New York Times, and 3.8M articles from Wikipedia. Stochastic inference can easily handle data sets of this size and outperforms traditional variational inference, which can only handle a smaller subset. (We also show that the Bayesian nonparametric topic model outperforms its parametric counterpart.) Stochastic variational inference lets us apply complex Bayesian models to massive data sets.
71
Equation of State Calculations by Fast Computing Machines
N. Metropolis, Arianna W. Rosenbluth, M. N. Rosenbluth et al. · The Journal of Chemical Physics · 1953 · 37.1K citations