2014 · 113 citations · 29 references
ComputingComputer ScienceComputational LinguisticsNatural Language ProcessingText AnalyticsCorpus AnalysisDocument RepresentationLanguage ModelingLanguage ModelsText MiningTopic ModelingMixture Models For TextSparse Document RepresentationSparse Topic ModelsTopic Theory
Topic modeling has been proved to be an effective method for exploratory text mining. It is a common assumption of most topic models that a document is generated from a mixture of topics. In real-world scenarios, individual documents usually concentrate on several salient topics instead of covering a wide variety of topics. A real topic also adopts a narrow range of terms instead of a wide coverage of the vocabulary. Understanding this sparsity of information is especially important for analyzing user-generated Web content and social media, which are featured as extremely short posts and condensed discussions. In this paper, we propose a dual-sparse topic model that addresses the sparsity in both the topic mixtures and the word usage. By applying a "Spike and Slab" prior to decouple the sparsity and smoothness of the document-topic and topic-word distributions, we allow individual documents to select a few focused topics and a topic to select focused terms, respectively. Experiments on different genres of large corpora demonstrate that the dual-sparse topic model outperforms both classical topic models and existing sparsity-enhanced topic models. This improvement is especially notable on collections of short documents.
29
Learning the parts of objects by non-negative matrix factorization
Daniel D. Lee, H. Sebastian Seung · Nature · 1999 · 14.1K citations
David M. Blei · Communications of the ACM · 2012 · 5.5K citations
Non-negative Matrix Factorization with Sparseness Constraints
Patrik O. Hoyer · 2004 · 2.6K citations