Are All Spanish Doctors Male? Evaluating Gender Bias in German Machine Translation

Michelle Kappl
2/26/2025
cs.CL

No confident code implementation found for this paper.

Abstract

We present WinoMTDE, a new gender bias evaluation test set designed to assess occupational stereotyping and underrepresentation in German machine translation (MT) systems. Building on the automatic evaluation method introduced by arXiv:1906.00591v1, we extend the approach to German, a language with grammatical gender. The WinoMTDE dataset comprises 288 German sentences that are balanced in regard to gender, as well as stereotype, which was annotated using German labor statistics. We conduct a large-scale evaluation of five widely used MT systems and a large language model. Our results reveal persistent bias in most models, with the LLM outperforming traditional systems. The dataset and evaluation code are publicly available under https://github.com/michellekappl/mt_gender_german.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. Any repos shown below are weak matches — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Exploring Gender Bias in Large Language Models: An In-depth Dive into the German Language

89% match2025

A Tale of Pronouns: Interpretability Informs Gender Bias Mitigation for Fairer Instruction-Tuned Machine Translation

88% match1 repo42023

What is Your Favorite Gender, MLM? Gender Bias Evaluation in Multilingual Masked Language Models

87% match2024

Building Bridges: A Dataset for Evaluating Gender-Fair Machine Translation into German

86% match1 repo2024

Gender Bias in MT for a Genderless Language: New Benchmarks for Basque

86% match2026

Assumed Identities: Quantifying Gender Bias in Machine Translation of Gender-Ambiguous Occupational Terms

86% match2025

Cite this paper

@article{kappl2025spanish,
  title  = {Are All Spanish Doctors Male? Evaluating Gender Bias in German Machine Translation},
  author = {Michelle Kappl},
  year   = {2025},
  eprint = {2502.19104},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2502.19104v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2502.19104.svg)](https://codeofpaper.com/paper/2502.19104)

Discussion