Dialect Normalization using Large Language Models and Morphological Rules

Antonios Dimakis, John Pavlopoulos, Antonios Anastasopoulos
6/10/2025
cs.CL

No official code found. Best implementation: andhmak/rule_dialnorm (1 star, last commit 2025-07-25).

Abstract

Natural language understanding systems struggle with low-resource languages, including many dialects of high-resource ones. Dialect-to-standard normalization attempts to tackle this issue by transforming dialectal text so that it can be used by standard-language tools downstream. In this study, we tackle this task by introducing a new normalization method that combines rule-based linguistically informed transformations and large language models (LLMs) with targeted few-shot prompting, without requiring any parallel data. We implement our method for Greek dialects and apply it on a dataset of regional proverbs, evaluating the outputs using human annotators. We then use this dataset to conduct downstream experiments, finding that previous results regarding these proverbs relied solely on superficial linguistic information, including orthographic artifacts, while new observations can still be made through the remaining semantics.

View on arXivView PDF

Code Implementations(1)

andhmak/rule_dialnormCommunity100%

Code and datasets associated with the paper titled "Dialect Normalization using Large Language Models and Morphological Rules"

10PythonFeb 11, 20251 years ago

Similar Papers

Natural Language Processing for Dialects of a Language: A Survey

84% match2024

Two Spelling Normalization Approaches Based on Large Language Models

83% match2025

Modeling Orthographic Variation in Occitan's Dialects

83% match2024

Data-Augmentation-Based Dialectal Adaptation for LLMs

83% match2024

Fine-Tuning LLMs for Low-Resource Dialect Translation: The Case of Lebanese

83% match2025

DialUp! Modeling the Language Continuum by Adapting Models to Dialects and Dialects to Models

83% match2025

Cite this paper

@article{dimakis2025dialect,
  title  = {Dialect Normalization using Large Language Models and Morphological Rules},
  author = {Antonios Dimakis and John Pavlopoulos and Antonios Anastasopoulos},
  year   = {2025},
  eprint = {2506.08907},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2506.08907v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2506.08907.svg)](https://codeofpaper.com/paper/2506.08907)

Discussion