AraVec: A set of Arabic Word Embedding Models for use in Arabic NLP

TLDR

Advances in neural networks have driven the development of word embeddings, which represent words as vectors that capture syntactic and semantic relationships. AraVec is an open‑source project that delivers free, high‑quality Arabic word embeddings for the NLP research community. The first AraVec release offers six embedding models trained on over 3.3 billion tokens from tweets, web pages, and Wikipedia, using extensive data cleaning, preprocessing, and state‑of‑the‑art embedding methods.

Abstract

Advancements in neural networks have led to developments in fields like computer vision, speech recognition and natural language processing (NLP). One of the most influential recent developments in NLP is the use of word embeddings, where words are represented as vectors in a continuous space, capturing many syntactic and semantic relations among them. AraVec is a pre-trained distributed word representation (word embedding) open source project which aims to provide the Arabic NLP research community with free to use and powerful word embedding models. The first version of AraVec provides six different word embedding models built on top of three different Arabic content domains; Tweets, World Wide Web pages and Wikipedia Arabic articles. The total number of tokens used to build the models amounts to more than 3,300,000,000. This paper describes the resources used for building the models, the employed data cleaning techniques, the carried out preprocessing step, as well as the details of the employed word embedding creation techniques.