2020 · 16 citations · 25 references
ComputingComputer ScienceData MiningStatistical ModelingText AnalyticsBayesian InferenceInferential StatisticsStatistical InferenceTopic ModelingVariational InferenceMonte Carlo SamplingNegative Binomial DistributionNegative Binomial ModelsNeural Topic ModelsParameter InferencePoisson DistributionTopic CoherenceTopic Discovery
Mixed counting models that use the negative binomial distribution as the prior can well model over-dispersed and hierarchically dependent random variables; thus they have attracted much attention in mining dispersed document topics. However, the existing parameter inference method like Monte Carlo sampling is quite time-consuming. In this paper, we propose two efficient neural mixed counting models, i.e., the Negative Binomial-Neural Topic Model (NB-NTM) and the Gamma Negative Binomial-Neural Topic Model (GNB-NTM) for dispersed topic discovery. Neural variational inference algorithms are developed to infer model parameters by using the reparameterization of Gamma distribution and the Gaussian approximation of Poisson distribution. Experiments on real-world datasets indicate that our models outperform state-of-the-art baseline models in terms of perplexity and topic coherence. The results also validate that both NB-NTM and GNB-NTM can produce explainable intermediate variables by generating dispersed proportions of document topics.
25
Indexing by latent semantic analysis
Scott Deerwester, Susan Dumais, George W. Furnas et al. · Journal of the American Society for Information Science · 1990 · 12.8K citations
David M. Blei, John Lafferty · 2006 · 2.4K citations
Probabilistic Latent Semantic Analysis
Thomas Hofmann · arXiv (Cornell University) · 2013 · 2.1K citations · Full text
Daniel Ramage, David Hall, Ramesh Nallapati et al. · 2009 · 1.2K citations · Full text