Arabizi Detection and Conversion to Arabic

TLDR

Arabizi is Arabic text written in Latin characters, used for both Modern Standard Arabic and dialects, and is common in informal social media where it is often mixed with English. The study aims to identify Arabizi in text and convert it to Arabic characters. The authors used word and sequence‑level features to detect Arabizi mixed with English and applied transliteration mining with language modeling to generate equivalent Arabic text. The approach achieved 98.5 % accuracy in identifying Arabizi and 88.7 % accuracy in converting it, with about a third of errors due to spelling and morphological variants.

Abstract

Arabizi is Arabic text that is written using Latin characters. Arabizi is used to present both Modern Standard Arabic (MSA) or Arabic dialects. It is commonly used in informal settings such as social networking sites and is often with mixed with English. In this paper we address the problems of: identifying Arabizi in text and converting it to Arabic characters. We used word and sequence-level features to identify Arabizi that is mixed with English. We achieved an identification accuracy of 98.5%. As for conversion, we used transliteration mining with language modeling to generate equivalent Arabic text. We achieved 88.7% conversion accuracy, with roughly a third of errors being spelling and morphological variants of the forms in ground truth.