Mathematical Foundations for a Compositional Distributional Model of\n Meaning

Abstract

We propose a mathematical framework for a unification of the distributional\ntheory of meaning in terms of vector space models, and a compositional theory\nfor grammatical types, for which we rely on the algebra of Pregroups,\nintroduced by Lambek. This mathematical framework enables us to compute the\nmeaning of a well-typed sentence from the meanings of its constituents.\nConcretely, the type reductions of Pregroups are `lifted' to morphisms in a\ncategory, a procedure that transforms meanings of constituents into a meaning\nof the (well-typed) whole. Importantly, meanings of whole sentences live in a\nsingle space, independent of the grammatical structure of the sentence. Hence\nthe inner-product can be used to compare meanings of arbitrary sentences, as it\nis for comparing the meanings of words in the distributional model. The\nmathematical structure we employ admits a purely diagrammatic calculus which\nexposes how the information flows between the words in a sentence in order to\nmake up the meaning of the whole sentence. A variation of our `categorical\nmodel' which involves constraining the scalars of the vector spaces to the\nsemiring of Booleans results in a Montague-style Boolean-valued semantics.\n