arXiv (Cornell University) · 2013 · 122 citations · 16 references
ComputingComputer ScienceComputational LinguisticsMachine LearningNatural Language ProcessingBayesian ComputationBayesian InferenceBayesian StatisticsStatistical InferenceStream ProcessingVariational InferenceBayesian Latent Dirichlet AllocationPosterior EstimationStochastic Variational InferenceVariational BayesVariational Bayes InferenceVariational Bayesian InferenceVariational Posterior Approximation
We present SDA-Bayes, a framework for (S)treaming, (D)istributed, (A)synchronous computation of a Bayesian posterior. The framework makes streaming updates to the estimated posterior according to a user-specified approximation batch primitive. We demonstrate the usefulness of our framework, with variational Bayes (VB) as the primitive, by fitting the latent Dirichlet allocation model to two large-scale document collections. We demonstrate the advantages of our algorithm over stochastic variational inference (SVI) by comparing the two after a single pass through a known amount of data---a case where SVI may be applied---and in the streaming setting, where SVI does not apply.
16
Stochastic variational inference
Matthew D. Hoffman, David M. Blei, Chong Wang et al. · Journal of Machine Learning Research · 2013 · 1.5K citations
Expectation Propagation for approximate Bayesian inference
Thomas P. Minka · arXiv (Cornell University) · 2013 · 1.5K citations · Full text
Online Learning for Latent Dirichlet Allocation
Matthew D. Hoffman, Francis Bach, David M. Blei · 2010 · 1.3K citations
HOGWILD!: A Lock-Free Approach to Parallelizing Stochastic Gradient Descent
Feng Niu, Benjamin Recht, Christopher Ré et al. · arXiv (Cornell University) · 2011 · 1.2K citations · Full text