Investigating Bias: A Multilingual Pipeline for Generating, Solving, and Evaluating Math Problems with LLMs

Mariam Mahran, Katharina Simbeck
9/22/2025
cs.CLcs.AIcs.LG

No confident code implementation found for this paper. 3 possible matches can be shown below.

Abstract

Large Language Models (LLMs) are increasingly used for educational support, yet their response quality varies depending on the language of interaction. This paper presents an automated multilingual pipeline for generating, solving, and evaluating math problems aligned with the German K-10 curriculum. We generated 628 math exercises and translated them into English, German, and Arabic. Three commercial LLMs (GPT-4o-mini, Gemini 2.5 Flash, and Qwen-plus) were prompted to produce step-by-step solutions in each language. A held-out panel of LLM judges, including Claude 3.5 Haiku, evaluated solution quality using a comparative framework. Results show a consistent gap, with English solutions consistently rated highest, and Arabic often ranked lower. These findings highlight persistent linguistic bias and the need for more equitable multilingual AI systems in education.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. 3 weaker matches are hidden by default — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Multilingual Performance Biases of Large Language Models in Education

87% match1 repo12025

Assessing Agentic Large Language Models in Multilingual National Bias

86% match2025

ChatGLM-Math: Improving Math Problem-Solving in Large Language Models with a Self-Critique Pipeline

85% match1 repo832024

MathRobust-LV: Evaluation of Large Language Models' Robustness to Linguistic Variations in Mathematical Reasoning

85% match2025

Cross-Language Bias Examination in Large Language Models

85% match2025

The AI Language Proficiency Monitor -- Tracking the Progress of LLMs on Multilingual Benchmarks

85% match2025

Cite this paper

@article{mahran2025investigating,
  title  = {Investigating Bias: A Multilingual Pipeline for Generating, Solving, and Evaluating Math Problems with LLMs},
  author = {Mariam Mahran and Katharina Simbeck},
  year   = {2025},
  eprint = {2509.17701},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2509.17701v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2509.17701.svg)](https://codeofpaper.com/paper/2509.17701)

Discussion