Constraint-Based Question Answering with Knowledge Graph

TLDR

WebQuestions and SimpleQuestions are benchmark datasets for KBQA, but they consist mainly of simple questions that can be answered with a single relation, limiting evaluation of systems on more complex queries. To address this gap, we introduce ComplexQuestions, a dataset of multi‑constraint questions that require multiple knowledge‑base relations to answer. We also present a novel systematic KBQA method designed to solve such multi‑constraint questions. Our method achieves comparable performance on WebQuestions and SimpleQuestions while delivering significant improvements on ComplexQuestions compared to state‑of‑the‑art baselines.

Abstract

WebQuestions and SimpleQuestions are two benchmark data-sets commonly used in recent knowledge-based question answering (KBQA) work. Most questions in them are ‘simple’ questions which can be answered based on a single relation in the knowledge base. Such data-sets lack the capability of evaluating KBQA systems on complicated questions. Motivated by this issue, we release a new data-set, namely ComplexQuestions, aiming to measure the quality of KBQA systems on ‘multi-constraint’ questions which require multiple knowledge base relations to get the answer. Beside, we propose a novel systematic KBQA approach to solve multi-constraint questions. Compared to state-of-the-art methods, our approach not only obtains comparable results on the two existing benchmark data-sets, but also achieves significant improvements on the ComplexQuestions.