Distant Supervision for Relation Extraction with an Incomplete Knowledge Base

TLDR

Distant supervision, which heuristically labels a corpus using a knowledge base, has become a popular method for training relation extractors. The study proposes an algorithm that learns from only positive and unlabeled labels at the pair‑of‑entity level. The algorithm builds on a state‑of‑the‑art distantly‑supervised extraction framework. It shows that many negative examples are false negatives because the knowledge base is incomplete, revealing a serious flaw in the heuristic, and experimental results demonstrate the proposed algorithm’s advantage over existing methods.

Abstract

Distant supervision, heuristically labeling a corpus using a knowledge base, has emerged as a popular choice for training relation extractors. In this paper, we show that a significant number of “negative“ examples generated by the labeling process are false negatives because the knowledge base is incomplete. Therefore the heuristic for generating negative examples has a seriousflaw. Building on a state-of-the-art distantly-supervised extraction algorithm, we proposed an algorithm that learns from only positive and unlabeled labels at the pair-of-entity level. Experimental results demonstrate its advantage over existing algorithms.