Distributional Memory: A General Framework for Corpus-Based Semantics

TLDR

Corpus‑based semantics research has traditionally employed ad hoc models that treat each task separately, extracting distinct distributional information from corpora. Distributional Memory extracts distributional information once into a weighted word‑link‑word tensor, then generates task‑specific matrices whose rows and columns form natural spaces for diverse semantic problems, enabling the same data to be shared across tasks such as word similarity, synonym discovery, categorization, selectional preference prediction, analogy solving, relation classification, qualia harvesting, property prediction, and verb alternation classification. Extensive empirical testing across these domains shows that Distributional Memory performs competitively against task‑specific algorithms and state‑of‑the‑art methods, demonstrating its viability despite its multi‑purpose nature.

Abstract

Research into corpus-based semantics has focused on the development of ad hoc models that treat single tasks, or sets of closely related tasks, as unrelated challenges to be tackled by extracting different kinds of distributional information from the corpus. As an alternative to this “one task, one model” approach, the Distributional Memory framework extracts distributional information once and for all from the corpus, in the form of a set of weighted word-link-word tuples arranged into a third-order tensor. Different matrices are then generated from the tensor, and their rows and columns constitute natural spaces to deal with different semantic problems. In this way, the same distributional information can be shared across tasks such as modeling word similarity judgments, discovering synonyms, concept categorization, predicting selectional preferences of verbs, solving analogy problems, classifying relations between word pairs, harvesting qualia structures with patterns or example pairs, predicting the typical properties of concepts, and classifying verbs into alternation classes. Extensive empirical testing in all these domains shows that a Distributional Memory implementation performs competitively against task-specific algorithms recently reported in the literature for the same tasks, and against our implementations of several state-of-the-art methods. The Distributional Memory approach is thus shown to be tenable despite the constraints imposed by its multi-purpose nature.