Discovery of inference rules for question-answering

TLDR

Question‑answering systems struggle with mismatched expressions between questions and texts, whereas humans rely on inference rules such as “X writes Y” implying “X is the author of Y”, which are generally unavailable to these systems due to construction difficulty. This paper proposes an unsupervised algorithm to automatically discover inference rules from text. The algorithm extends Harris’ Distributional Hypothesis to dependency‑tree paths, assuming that paths linking the same set of words have similar meanings. Illustrative examples demonstrate that the system uncovers numerous inference rules that humans often overlook.

Abstract

One of the main challenges in question-answering is the potential mismatch between the expressions in questions and the expressions in texts. While humans appear to use inference rules such as ‘X writes Y’ implies ‘X is the author of Y’ in answering questions, such rules are generally unavailable to question-answering systems due to the inherent difficulty in constructing them. In this paper, we present an unsupervised algorithm for discovering inference rules from text. Our algorithm is based on an extended version of Harris’ Distributional Hypothesis, which states that words that occurred in the same contexts tend to be similar. Instead of using this hypothesis on words, we apply it to paths in the dependency trees of a parsed corpus. Essentially, if two paths tend to link the same set of words, we hypothesize that their meanings are similar. We use examples to show that our system discovers many inference rules easily missed by humans.