StereoSet: Measuring stereotypical bias in pretrained language models

Moin Nadeem, Anna Bethke, Siva Reddy
4/20/2020
cs.CLcs.AIcs.CY

Official code: moinnadeem/StereoSet (206 stars, CC-BY-SA-4.0, last commit 2022-12-08).

Abstract

A stereotype is an over-generalized belief about a particular group of people, e.g., Asians are good at math or Asians are bad drivers. Such beliefs (biases) are known to hurt target groups. Since pretrained language models are trained on large real world data, they are known to capture stereotypical biases. In order to assess the adverse effects of these models, it is important to quantify the bias captured in them. Existing literature on quantifying bias evaluates pretrained language models on a small set of artificially constructed bias-assessing sentences. We present StereoSet, a large-scale natural dataset in English to measure stereotypical biases in four domains: gender, profession, race, and religion. We evaluate popular models like BERT, GPT-2, RoBERTa, and XLNet on our dataset and show that these models exhibit strong stereotypical biases. We also present a leaderboard with a hidden test set to track the bias of future language models at https://stereoset.mit.edu

View on arXivView PDF

Code Implementations(1)

moinnadeem/StereoSetOfficial100%
CC-BY-SA-4.0

StereoSet: Measuring stereotypical bias in pretrained language models

20647Shell, PythonApr 21, 20203 years agoCC-BY-SA-4.0

Similar Papers

Bias A-head? Analyzing Bias in Transformer-Based Language Model Attention Heads

89% match2023

Stereotype Detection as a Catalyst for Enhanced Bias Detection: A Multi-Task Learning Approach

89% match2025

Social Bias Probing: Fairness Benchmarking for Language Models

89% match2023

Indian-BhED: A Dataset for Measuring India-Centric Biases in Large Language Models

88% match1 repo42023

Bias and Volatility: A Statistical Framework for Evaluating Large Language Model's Stereotypes and the Associated Generation Inconsistency

88% match1 repo32024

Gender bias and stereotypes in Large Language Models

88% match2023

Cite this paper

@article{nadeem2020stereoset,
  title  = {StereoSet: Measuring stereotypical bias in pretrained language models},
  author = {Moin Nadeem and Anna Bethke and Siva Reddy},
  year   = {2020},
  eprint = {2004.09456},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/2004.09456}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2004.09456.svg)](https://codeofpaper.com/paper/2004.09456)

Discussion