On Aligning OpenIE Extractions with Knowledge Bases: A Case Study

Abstract

Open information extraction (OIE) is the task of extracting relations and their corresponding arguments from natural language text in unsupervised manner. Outputs of such systems are used for downstream tasks such as question answering and automatic knowledge base (KB) construction. Many of these downstream tasks rely on aligning OIE triples with reference KBs. Such alignments are usually evaluated w.r.t. a specific downstream task and, to date, no direct manual evaluation of such alignments has been performed. In this paper, we directly evaluate how OIE triples from the OPIEC corpus are related to the DBpedia KB w.r.t. information content. First, we investigate OPIEC triples and DBpedia facts having the same arguments by comparing the information on the OIE surface relation with the KB relation. Second, we evaluate the expressibility of general OPIEC triples in DBpedia. We investigate whether-and, if so, how-a given OIE triple can be mapped to a single KB fact. We found that such mappings are not always possible because the information in the OIE triples tends to be more specific. Our evaluation suggests, however, that significant part of OIE triples can be expressed by means of KB formulas instead of individual facts.