GATE: A Challenge Set for Gender-Ambiguous Translation Examples

Spencer Rarrick, Ranjita Naik, Varun Mathur, Sundar Poudel, Vishal Chowdhary
3/7/2023
cs.CL

No official code found. Best implementation: MicrosoftTranslator/GATE (4 stars, MIT, last commit 2023-03-16).

Abstract

Although recent years have brought significant progress in improving translation of unambiguously gendered sentences, translation of ambiguously gendered input remains relatively unexplored. When source gender is ambiguous, machine translation models typically default to stereotypical gender roles, perpetuating harmful bias. Recent work has led to the development of "gender rewriters" that generate alternative gender translations on such ambiguous inputs, but such systems are plagued by poor linguistic coverage. To encourage better performance on this task we present and release GATE, a linguistically diverse corpus of gender-ambiguous source sentences along with multiple alternative target language translations. We also provide tools for evaluation and system analysis when using GATE and use them to evaluate our translation rewriter system.

View on arXivView PDF

Code Implementations(1)

MicrosoftTranslator/GATECommunity100%
MIT

GATE: A Challenge Set for Gender-Ambiguous Translation Examples

41PythonFeb 24, 20233 years agoMIT

Similar Papers

Evaluating Gender Bias in the Translation of Gender-Neutral Languages into English

92% match2023

ConGA: Guidelines for Contextual Gender Annotation. A Framework for Annotating Gender in Machine Translation

91% match2026

Gender-specific Machine Translation with Large Language Models

91% match2023

Gender Bias in MT for a Genderless Language: New Benchmarks for Basque

90% match2026

How To Build Competitive Multi-gender Speech Translation Models For Controlling Speaker Gender Translation

89% match1 repo602023

Don't Overlook the Grammatical Gender: Bias Evaluation for Hindi-English Machine Translation

88% match2023

Cite this paper

@article{rarrick2023gate,
  title  = {GATE: A Challenge Set for Gender-Ambiguous Translation Examples},
  author = {Spencer Rarrick and Ranjita Naik and Varun Mathur and Sundar Poudel and Vishal Chowdhary},
  year   = {2023},
  eprint = {2303.03975},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2303.03975v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2303.03975.svg)](https://codeofpaper.com/paper/2303.03975)

Discussion