Ara-HOPE: Human-Centric Post-Editing Evaluation for Dialectal Arabic to Modern Standard Arabic Translation

Abdullah Alabdullah, Lifeng Han, Chenghua Lin
12/25/2025
cs.CL

Official code: abdullahalabdullah/Ara-HOPE (0 stars, last commit 2026-02-22).

Abstract

Dialectal Arabic to Modern Standard Arabic (DA-MSA) translation is a challenging task in Machine Translation (MT) due to significant lexical, syntactic, and semantic divergences between Arabic dialects and MSA. Existing automatic evaluation metrics and general-purpose human evaluation frameworks struggle to capture dialect-specific MT errors, hindering progress in translation assessment. This paper introduces Ara-HOPE, a human-centric post-editing evaluation framework designed to systematically address these challenges. The framework includes a five-category error taxonomy and a decision-tree annotation protocol. Through comparative evaluation of three MT systems (Arabic-centric Jais, general-purpose GPT-3.5, and baseline NLLB-200), Ara-HOPE effectively highlights systematic performance differences between these systems. Our results show that dialect-specific terminology and semantic preservation remain the most persistent challenges in DA-MSA translation. Ara-HOPE establishes a new framework for evaluating Dialectal Arabic MT quality and provides actionable guidance for improving dialect-aware MT systems. For reproducibility, we make the annotation files and related materials publicly available at https://github.com/abdullahalabdullah/Ara-HOPE

View on arXivView PDF

Code Implementations(1)

abdullahalabdullah/Ara-HOPEOfficial100%

This repository contains the excel file used for annotations in our paper: "Ara-HOPE: Human-Centric Post-Editing Evaluation for Dialectal Arabic to Modern Standard Arabic Translation”. The paper can be found at : [https://www.arxiv.org/abs/2512.21787](https://www.arxiv.org/abs/2512.21787)

00Feb 7, 20268 months ago

Similar Papers

Advancing Dialectal Arabic to Modern Standard Arabic Machine Translation

88% match1 repo2025

AraDiCE: Benchmarks for Dialectal and Cultural Capabilities in LLMs

86% match2024

Alexandria: A Multi-Domain Dialectal Arabic Machine Translation Dataset for Culturally Inclusive and Linguistically Diverse LLMs

83% match2026

SHAMI-MT: A Syrian Arabic Dialect to Modern Standard Arabic Bidirectional Machine Translation System

82% match2025

From Guidelines to Practice: A New Paradigm for Arabic Language Model Evaluation

82% match2025

ArbESC+: Arabic Enhanced Edit Selection System Combination for Grammatical Error Correction Resolving conflict and improving system combination in Arabic GEC

81% match2025

Cite this paper

@article{alabdullah2025arahope,
  title  = {Ara-HOPE: Human-Centric Post-Editing Evaluation for Dialectal Arabic to Modern Standard Arabic Translation},
  author = {Abdullah Alabdullah and Lifeng Han and Chenghua Lin},
  year   = {2025},
  eprint = {2512.21787},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2512.21787v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2512.21787.svg)](https://codeofpaper.com/paper/2512.21787)

Discussion