Topics in semantic representation.

TLDR

Processing language requires retrieving concepts from memory during a continuous stream of information, and this retrieval is aided by inferring the gist of a sentence, conversation, or document to predict related concepts and disambiguate words. The article analyzes the computational problem of extracting and using gist, formulating it as rational statistical inference. This leads to a novel semantic representation in which word meanings are encoded as probabilistic topics. The topic model accurately predicts word association and the effects of semantic association and ambiguity on language‑processing and memory tasks, and it provides a foundation for richer statistical models of language that can incorporate additional semantic and syntactic structure.

Abstract

Processing language requires the retrieval of concepts from memory in response to an ongoing stream of information. This retrieval is facilitated if one can infer the gist of a sentence, conversation, or document and use that gist to predict related concepts and disambiguate words. This article analyzes the abstract computational problem underlying the extraction and use of gist, formulating this problem as a rational statistical inference. This leads to a novel approach to semantic representation in which word meanings are represented in terms of a set of probabilistic topics. The topic model performs well in predicting word association and the effects of semantic association and ambiguity on a variety of language-processing and memory tasks. It also provides a foundation for developing more richly structured statistical models of language, as the generative process assumed in the topic model can easily be extended to incorporate other kinds of semantic and syntactic structure.