Finding scientific topics

TLDR

Identifying a document’s content begins by determining its topics, and this paper builds on the generative topic model introduced by Blei, Ng, and Jordan, where each document is generated by selecting a topic distribution and sampling words from topics according to that distribution. The authors employ a Markov chain Monte Carlo inference algorithm for the LDA model, applying it to PNAS abstracts and using Bayesian model selection to determine the optimal number of topics. The extracted topics reveal meaningful structure aligned with authors’ class designations, and the analysis enables further applications such as detecting hot topics via temporal dynamics and tagging abstracts to illustrate semantic content. Reference: Blei, Ng, and Jordan (2003), J.

Abstract

A first step in identifying the content of a document is determining which topics that document addresses. We describe a generative model for documents, introduced by Blei, Ng, and Jordan [Blei, D. M., Ng, A. Y. & Jordan, M. I. (2003) J. Machine Learn. Res. 3, 993-1022], in which each document is generated by choosing a distribution over topics and then choosing each word in the document from a topic selected according to this distribution. We then present a Markov chain Monte Carlo algorithm for inference in this model. We use this algorithm to analyze abstracts from PNAS by using Bayesian model selection to establish the number of topics. We show that the extracted topics capture meaningful structure in the data, consistent with the class designations provided by the authors of the articles, and outline further applications of this analysis, including identifying “hot topics” by examining temporal dynamics and tagging abstracts to illustrate semantic content.