My LLM might Mimic AAE -- But When Should it?

Sandra C. Sandoval, Christabel Acquaye, Kwesi Cobbina, Mohammad Nayeem Teli, Hal Daumé
2/6/2025
cs.CL

Official code: smelliecat/AAEMime (1 star, MIT, last commit 2025-02-15).

Abstract

We examine the representation of African American English (AAE) in large language models (LLMs), exploring (a) the perceptions Black Americans have of how effective these technologies are at producing authentic AAE, and (b) in what contexts Black Americans find this desirable. Through both a survey of Black Americans ($n=$ 104) and annotation of LLM-produced AAE by Black Americans ($n=$ 228), we find that Black Americans favor choice and autonomy in determining when AAE is appropriate in LLM output. They tend to prefer that LLMs default to communicating in Mainstream U.S. English in formal settings, with greater interest in AAE production in less formal settings. When LLMs were appropriately prompted and provided in context examples, our participants found their outputs to have a level of AAE authenticity on par with transcripts of Black American speech. Select code and data for our project can be found here: https://github.com/smelliecat/AAEMime.git

View on arXivView PDF

Code Implementations(1)

smelliecat/AAEMimeOfficial100%
MIT
10Shell, PythonJan 27, 20251 years agoMIT

Similar Papers

Evaluating the Usage of African-American Vernacular English in Large Language Models

89% match2026

Disparities in LLM Reasoning Accuracy and Explanations: A Case Study on African American English

85% match1 repo52025

Data Caricatures: On the Representation of African American Language in Pretraining Corpora

85% match1 repo2025

Analysis of LLM as a grammatical feature tagger for African American English

84% match2025

AAVENUE: Detecting LLM Biases on NLU Tasks in AAVE via a Novel Benchmark

83% match2024

Rejected Dialects: Biases Against African American Language in Reward Models

82% match1 repo2025

Cite this paper

@article{sandoval2025might,
  title  = {My LLM might Mimic AAE -- But When Should it?},
  author = {Sandra C. Sandoval and Christabel Acquaye and Kwesi Cobbina and Mohammad Nayeem Teli and Hal Daumé},
  year   = {2025},
  eprint = {2502.04564},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2502.04564v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2502.04564.svg)](https://codeofpaper.com/paper/2502.04564)

Discussion