ANERsys 2.0: Conquering the NER Task for the Arabic Language by Combining the Maximum Entropy with POS-tag Information

Abstract

In this paper we describe an improved version of ANERsys, an Arabic Named Entity Recognition system for open-domain texts. The first version of ANERsys was totally based on the Maximum Entropy ap- proach and was trained and tested with corpora which we have built our- selves. The results showed that the Maximum Entropy is an appropriate method to identify Named Entities in Arabic texts. However, in order to reach higher performance a greater eort needed to be done to improve the recognition of long proper names. Therefore, in the second version of ANERsys, we use a Part Of Speech tagger and a two-steps approach to enhance the performance of the system. Furthermore, we have used our own (now freely available on our website) corpora (ANERcorp) and gazetteers (ANERgazet) to train and evaluate ANERsys 2.0. We carried out several experiments to evaluate the performance of the system and to compare it with the online freely available demo version of the com- mercial system Siraj (Sakhr). The results show that the accuracy of the new version is 10 points above the old one and more than 7 points above the Siraj (Sakhr) system accuracy.