A Group Fairness Lens for Large Language Models

Guanqun Bi, Yuqiang Xie, Lei Shen, Yanan Cao
12/24/2023
cs.CL

Official code: surika/Group-Fairness-LLMs (0 stars, last commit 2025-09-11).

Abstract

The need to assess LLMs for bias and fairness is critical, with current evaluations often being narrow, missing a broad categorical view. In this paper, we propose evaluating the bias and fairness of LLMs from a group fairness lens using a novel hierarchical schema characterizing diverse social groups. Specifically, we construct a dataset, GFAIR, encapsulating target-attribute combinations across multiple dimensions. Moreover, we introduce statement organization, a new open-ended text generation task, to uncover complex biases in LLMs. Extensive evaluations of popular LLMs reveal inherent safety concerns. To mitigate the biases of LLMs from a group fairness perspective, we pioneer a novel chainof-thought method GF-THINK to mitigate biases of LLMs from a group fairness perspective. Experimental results demonstrate its efficacy in mitigating bias and achieving fairness in LLMs. Our dataset and codes are available at https://github.com/surika/Group-Fairness-LLMs.

View on arXivView PDF

Code Implementations(1)

surika/Group-Fairness-LLMsOfficial100%

This repository contains the official implementation of the paper "A Group Fairness Lens for Large Language Models" published in Findings of EMNLP 2025.

00Shell, PythonSep 11, 20251 years ago

Similar Papers

Bias and Fairness in Large Language Models: A Survey

87% match1 repo1622023

Fairness Evaluation in Text Classification: Machine Learning Practitioner Perspectives of Individual and Group Fairness

86% match2023

Protected group bias and stereotypes in Large Language Models

85% match2024

Social Bias Probing: Fairness Benchmarking for Language Models

84% match2023

SocialStigmaQA: A Benchmark to Uncover Stigma Amplification in Generative Language Models

84% match2023

BiasTestGPT: Using ChatGPT for Social Bias Testing of Language Models

84% match2023

Cite this paper

@article{bi2023group,
  title  = {A Group Fairness Lens for Large Language Models},
  author = {Guanqun Bi and Yuqiang Xie and Lei Shen and Yanan Cao},
  year   = {2023},
  eprint = {2312.15478},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2312.15478v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2312.15478.svg)](https://codeofpaper.com/paper/2312.15478)

Discussion