Evaluating Gender Bias in the Translation of Gender-Neutral Languages into English

Spencer Rarrick, Ranjita Naik, Sundar Poudel, Vishal Chowdhary
11/15/2023
cs.CLcs.AI

No confident code implementation found for this paper. 4 possible matches can be shown below.

Abstract

Machine Translation (MT) continues to improve in quality and adoption, yet the inadvertent perpetuation of gender bias remains a significant concern. Despite numerous studies into gender bias in translations from gender-neutral languages such as Turkish into more strongly gendered languages like English, there are no benchmarks for evaluating this phenomenon or for assessing mitigation strategies. To address this gap, we introduce GATE X-E, an extension to the GATE (Rarrick et al., 2023) corpus, that consists of human translations from Turkish, Hungarian, Finnish, and Persian into English. Each translation is accompanied by feminine, masculine, and neutral variants for each possible gender interpretation. The dataset, which contains between 1250 and 1850 instances for each of the four language pairs, features natural sentences with a wide range of sentence lengths and domains, challenging translation rewriters on various linguistic phenomena. Additionally, we present an English gender rewriting solution built on GPT-3.5 Turbo and use GATE X-E to evaluate it. We open source our contributions to encourage further research on gender debiasing.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. 4 weaker matches are hidden by default — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

GATE: A Challenge Set for Gender-Ambiguous Translation Examples

92% match1 repo42023

ConGA: Guidelines for Contextual Gender Annotation. A Framework for Annotating Gender in Machine Translation

91% match2026

Gender Bias in MT for a Genderless Language: New Benchmarks for Basque

90% match2026

Gender-specific Machine Translation with Large Language Models

89% match2023

Don't Overlook the Grammatical Gender: Bias Evaluation for Hindi-English Machine Translation

89% match2023

Gender Inflected or Bias Inflicted: On Using Grammatical Gender Cues for Bias Evaluation in Machine Translation

89% match2023

Cite this paper

@article{rarrick2023evaluating,
  title  = {Evaluating Gender Bias in the Translation of Gender-Neutral Languages into English},
  author = {Spencer Rarrick and Ranjita Naik and Sundar Poudel and Vishal Chowdhary},
  year   = {2023},
  eprint = {2311.08836},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2311.08836v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2311.08836.svg)](https://codeofpaper.com/paper/2311.08836)

Discussion