Assessing the Political Fairness of Multilingual LLMs: A Case Study based on a 21-way Multiparallel EuroParl Dataset

Paul Lerner, François Yvon
10/23/2025
cs.CL

No official code found. Best implementation: PaulLerner/21-EuroParl (1 star, MIT, last commit 2025-10-24).

Abstract

The political biases of Large Language Models (LLMs) are usually assessed by simulating their answers to English surveys. In this work, we propose an alternative framing of political biases, relying on principles of fairness in multilingual translation. We systematically compare the translation quality of speeches in the European Parliament (EP), observing systematic differences with majority parties from left, center, and right being better translated than outsider parties. This study is made possible by a new, 21-way multiparallel version of EuroParl, the parliamentary proceedings of the EP, which includes the political affiliations of each speaker. The dataset consists of 1.5M sentences for a total of 40M words and 249M characters. It covers three years, 1000+ speakers, 7 countries, 12 EU parties, 25 EU committees, and hundreds of national parties.

View on arXivView PDF

Code Implementations(1)

PaulLerner/21-EuroParlCommunity60%
MIT

Dataset and code for the paper "Assessing the Political Fairness of Multilingual LLMs: A Case Study based on a 21-way Multiparallel EuroParl Dataset" (Lerner and Yvon, 2025)

10Python, Jupyter NotebookOct 16, 202512 months agoMIT

Similar Papers

Bias Beyond Borders: Political Ideology Evaluation and Steering in Multilingual LLMs

87% match2026

A Systematic Analysis of Biases in Large Language Models

86% match2025

Uncovering Political Bias in Large Language Models using Parliamentary Voting Records

86% match2026

Assessing Political Bias in Large Language Models

85% match2024

Do Political Opinions Transfer Between Western Languages? An Analysis of Unaligned and Aligned Multilingual LLMs

85% match2025

Benchmarking Gender and Political Bias in Large Language Models

85% match2025

Cite this paper

@article{lerner2025assessing,
  title  = {Assessing the Political Fairness of Multilingual LLMs: A Case Study based on a 21-way Multiparallel EuroParl Dataset},
  author = {Paul Lerner and François Yvon},
  year   = {2025},
  eprint = {2510.20508},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2510.20508v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2510.20508.svg)](https://codeofpaper.com/paper/2510.20508)

Discussion