The CoNLL-2008 shared task on joint parsing of syntactic and semantic dependencies

TLDR

The annual shared task of the Conference on Computational Natural Language Learning promotes NLP applications and evaluates them in a standard setting, and in 2008 it focused on joint parsing of syntactic and semantic dependencies. The paper defines the shared task, explains dataset creation, and reports and analyzes participating systems’ results. The task unifies prior shared tasks under a single dependency‑based formalism, adding named‑entity boundaries to syntactic dependencies and modeling roles of verbal and nominal predicates in semantic dependencies. The results show the performance of participating systems and provide analysis of their approaches.

Abstract

The Conference on Computational Natural Language Learning is accompanied every year by a shared task whose purpose is to promote natural language processing applications and evaluate them in a standard setting. In 2008 the shared task was dedicated to the joint parsing of syntactic and semantic dependencies. This shared task not only unifies the shared tasks of the previous four years under a unique dependency-based formalism, but also extends them significantly: this year's syntactic dependencies include more information such as named-entity boundaries; the semantic dependencies model roles of both verbal and nominal predicates. In this paper, we define the shared task and describe how the data sets were created. Furthermore, we report and analyze the results and describe the approaches of the participating systems.