Multi-EuP: The Multilingual European Parliament Dataset for Analysis of Bias in Information Retrieval

Jinrui Yang, Timothy Baldwin, Trevor Cohn
11/3/2023
cs.CLcs.AIcs.IR

No confident code implementation found for this paper.

Abstract

We present Multi-EuP, a new multilingual benchmark dataset, comprising 22K multi-lingual documents collected from the European Parliament, spanning 24 languages. This dataset is designed to investigate fairness in a multilingual information retrieval (IR) context to analyze both language and demographic bias in a ranking context. It boasts an authentic multilingual corpus, featuring topics translated into all 24 languages, as well as cross-lingual relevance judgments. Furthermore, it offers rich demographic information associated with its documents, facilitating the study of demographic bias. We report the effectiveness of Multi-EuP for benchmarking both monolingual and multilingual IR. We also conduct a preliminary experiment on language bias caused by the choice of tokenization strategy.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. Any repos shown below are weak matches — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Assessing the Political Fairness of Multilingual LLMs: A Case Study based on a 21-way Multiparallel EuroParl Dataset

84% match1 repo12025

Language Fairness in Multilingual Information Retrieval

83% match1 repo32024

Investigating Language and Retrieval Bias in Multilingual Previously Fact-Checked Claim Detection

82% match1 repo42025

Language Bias in Information Retrieval: The Nature of the Beast and Mitigation Methods

82% match2025

EuroSpeech: A Multilingual Speech Corpus

81% match2025

Not All Languages are Equal: Insights into Multilingual Retrieval-Augmented Generation

80% match1 repo12024

Cite this paper

@article{yang2023multieup,
  title  = {Multi-EuP: The Multilingual European Parliament Dataset for Analysis of Bias in Information Retrieval},
  author = {Jinrui Yang and Timothy Baldwin and Trevor Cohn},
  year   = {2023},
  eprint = {2311.01870},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2311.01870v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2311.01870.svg)](https://codeofpaper.com/paper/2311.01870)

Discussion