CoNLL-X shared task on multilingual dependency parsing

TLDR

The CoNLL shared task provides a common benchmark for evaluating multilingual dependency parsers, and the tenth iteration focused on parsing across 13 languages, summarizing participants’ approaches and results. This study converts 13 language treebanks into a unified dependency format, measures parsing performance, and seeks to identify factors that influence multilingual parsing difficulty. The authors standardize treebanks into a common dependency format and evaluate parsing accuracy across languages.

Abstract

Each year the Conference on Computational Natural Language Learning (CoNLL) features a shared task, in which participants train and test their systems on exactly the same data sets, in order to better compare systems. The tenth CoNLL (CoNLL-X) saw a shared task on Multilingual Dependency Parsing. In this paper, we describe how treebanks for 13 languages were converted into the same dependency format and how parsing performance was measured. We also give an overview of the parsing approaches that participants took and the results that they achieved. Finally, we try to draw general conclusions about multi-lingual parsing: What makes a particular language, treebank or annotation scheme easier or harder to parse and which phenomena are challenging for any dependency parser?