SemEval-2022 Task 11: Multilingual Complex Named Entity Recognition (MultiCoNER)

TLDR

The task comprised 13 tracks, including monolingual, multilingual, and code‑mixed NER models, evaluated on the MULTICONER dataset of 2.3 million instances across 11 languages. Methods that fused external knowledge into transformer models achieved the highest performance, yet recognizing creative works remains difficult, and the competition attracted 377 participants in practice and 55 system submissions in the final phase.

Abstract

We present the findings of SemEval-2022 Task 11 on Multilingual Complex Named Entity Recognition MULTICONER. Divided into 13 tracks, the task focused on methods to identify complex named entities (like names of movies, products and groups) in 11 languages in both monolingual and multi-lingual scenarios. Eleven tracks required building monolingual NER models for individual languages, one track focused on multilingual models able to work on all languages, and the last track featured code-mixed texts within any of these languages. The task is based on the MULTICONER dataset comprising of 2.3 millions instances in Bangla, Chinese, Dutch, English, Farsi, German, Hindi, Korean, Russian, Spanish, and Turkish. Results showed that methods fusing external knowledge into transformer models achieved the best results. However, identifying entities like creative works is still challenging even with external knowledge. MULTICONER was one of the most popular tasks in SemEval-2022 and it attracted 377 participants during the practice phase. 236 participants signed up for the final test phase and 55 teams submitted their systems.