Arabic Text Diacritization Using Deep Neural Networks

Ali Fadel, Ibraheem Tuffaha, Bara' Al-Jawarneh, Mahmoud Al-Ayyoub
4/25/2019
cs.CLcs.LG

Official code: Barqawiz/Shakkala (355 stars, last commit 2023-03-25). 1 more implementation below.

Abstract

Diacritization of Arabic text is both an interesting and a challenging problem at the same time with various applications ranging from speech synthesis to helping students learning the Arabic language. Like many other tasks or problems in Arabic language processing, the weak efforts invested into this problem and the lack of available (open-source) resources hinder the progress towards solving this problem. This work provides a critical review for the currently existing systems, measures and resources for Arabic text diacritization. Moreover, it introduces a much-needed free-for-all cleaned dataset that can be easily used to benchmark any work on Arabic diacritization. Extracted from the Tashkeela Corpus, the dataset consists of 55K lines containing about 2.3M words. After constructing the dataset, existing tools and systems are tested on it. The results of the experiments show that the neural Shakkala system significantly outperforms traditional rule-based approaches and other closed-source tools with a Diacritic Error Rate (DER) of 2.88% compared with 13.78%, which the best DER for the non-neural approach (obtained by the Mishkal tool).

View on arXivView PDF

Code Implementations(2)

Barqawiz/ShakkalaOfficial100%

Deep learning for AR text Vocalization - التشكيل الالي للنصوص العربية

35543PythonNov 28, 20173 years agoNOASSERTION
arabic-nlpdeep-learninglstmrnn
AliOsm/arabic-text-diacritizationOfficial100%
MIT

Benchmark Arabic text diacritization dataset

8222Python, Jupyter NotebookFeb 22, 20196 months agoMIT
arabic-languagecomparisondatasetdiacritizationiccais+2 more

Similar Papers

Neural Arabic Text Diacritization: State of the Art Results and a Novel Approach for Machine Translation

86% match2 repos992019

Diacritic Recognition Performance in Arabic ASR

85% match2023

Fine-Tashkeel: Finetuning Byte-Level Models for Accurate Arabic Text Diacritization

85% match2023

Take the Hint: Improving Arabic Diacritization with Partially-Diacritized Text

84% match1 repo42023

Automatic Restoration of Diacritics for Speech Data Sets

82% match1 repo92023

A User-Centered Evaluation of Spanish Text Simplification

82% match1 repo42023

Cite this paper

@article{fadel2019arabic,
  title  = {Arabic Text Diacritization Using Deep Neural Networks},
  author = {Ali Fadel and Ibraheem Tuffaha and Bara' Al-Jawarneh and Mahmoud Al-Ayyoub},
  year   = {2019},
  eprint = {1905.01965},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/1905.01965}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/1905.01965.svg)](https://codeofpaper.com/paper/1905.01965)

Discussion