2006 · 57 citations · 3 references
Arabic Dialect LinguisticsEngineeringArabic Morphological AnalysisArabic OrthographyCorpus LinguisticsText MiningNatural Language ProcessingArabic Text SimplificationString-searching AlgorithmArabicText SegmentationHidden Markov ModelComputational LinguisticsString ProcessingArabic Dialect OrthographyLanguage StudiesArabic ReadabilityMachine TranslationArabic Syntactic AnalysisStatistical MethodsArabic Dialect Morphological AnalysisText NormalizationText ProcessingLanguage ModelingLinguistics
Undiacritized Arabic text is treated as an observation sequence in a hidden Markov model, with hidden states representing possible diacritized word forms. The paper investigates adding diacritics to undiacritized Arabic text using statistical language modeling. A large fully diacritized corpus is used to extract monogram, bigram, and trigram language models, and a hidden Markov model with Viterbi decoding is applied to find the most probable diacritized word sequence. Evaluation on Qur’an text demonstrates the algorithm’s performance and suggests ways to improve it.
In this paper, the issue of adding diacritics Tashkeel to undiacritized Arabic text using statistical methods for language modeling is addressed. The approach requires a large corpus of fully diacritized text for extracting the language monograms, bigrams, and trigrams for words and letters. Search algorithms are then used o find the best probable sequence of diacritized words of a given undiacritized word sequence. The word sequence of undiacritized Arabic text is considered an observation sequence from a hidden Markov Model, where the hidden states are the possible diacritized expressions of the words. The optimal sequence of diacritized words (or states) is then efficiently obtained using Viterbi Algorithm. We present an evaluation of the basic algorithm using the Qur’an’s text, and discuss various ramifications for improving the performance of this approach.
3
Techniques for high quality Arabic speech synthesis
Moustafa Elshafei · Information Sciences · 2002 · 56 citations