Concepedia

The State and Fate of Linguistic Diversity and Inclusion in the NLP World

Pratik Joshi, Sebastin Santy, Amar Budhiraja, Kalika Bali, Monojit Choudhury

2020 · 40 citations · 10 references

Open access

TL;DR

Language technologies aim to promote multilingualism and linguistic diversity, yet only a tiny fraction of the world’s 7000+ languages are represented in rapidly evolving language technologies and applications. The paper examines how language types and resource availability influence their representation in NLP conferences, and urges the ACL community to prioritize addressing these disparities so no language is left behind. The study analyzes the relationship between language types, resource availability, and conference representation to trace each language’s trajectory over time. The quantitative analysis reveals a stark disparity in language resources and questions the purported language‑agnostic nature of current NLP models and systems.

Abstract

Language technologies contribute to promoting multilingualism and linguistic diversity around the world. However, only a very small number of the over 7000 languages of the world are represented in the rapidly evolving language technologies and applications. In this paper we look at the relation between the types of languages, resources, and their representation in NLP conferences to understand the trajectory that different languages have followed over time. Our quantitative investigation underlines the disparity between languages, especially in terms of their resources, and calls into question the “language agnostic” status of current models and systems. Through this paper, we attempt to convince the ACL community to prioritise the resolution of the predicaments highlighted here, so that no language is left behind.