Unsupervised Concept Vector Extraction for Bias Control in LLMs

Hannah Cyberey, Yangfeng Ji, David Evans
2/27/2025
cs.CLcs.CY

Official code: hannahxchen/gender-bias-steering (2 stars, MIT, last commit 2026-04-07).

Abstract

Large language models (LLMs) are known to perpetuate stereotypes and exhibit biases. Various strategies have been proposed to mitigate these biases, but most work studies biases as a black-box problem without considering how concepts are represented within the model. We adapt techniques from representation engineering to study how the concept of "gender" is represented within LLMs. We introduce a new method that extracts concept representations via probability weighting without labeled data and efficiently selects a steering vector for measuring and manipulating the model's representation. We develop a projection-based method that enables precise steering of model predictions and demonstrate its effectiveness in mitigating gender bias in LLMs and show that it also generalizes to racial bias. Our code is available at: https://github.com/hannahxchen/gender-bias-steering

View on arXivView PDF

Code Implementations(1)

hannahxchen/gender-bias-steeringOfficial100%
MIT

Code for the paper "Unsupervised Concept Vector Extraction for Bias Control in LLMs"

21Python, Jupyter NotebookJan 4, 20256 months agoMIT

Similar Papers

Disclosure and Mitigation of Gender Bias in LLMs

88% match1 repo32024

Mitigating Gender Bias via Fostering Exploratory Thinking in LLMs

86% match1 repo22025

Shifting Perspectives: Steering Vectors for Robust Bias Mitigation in LLMs

85% match2 repos82025

BiasGym: Fantastic LLM Biases and How to Find (and Remove) Them

85% match2025

Decoding Biases: Automated Methods and LLM Judges for Gender Bias Detection in Language Models

85% match2024

Dissecting Bias in LLMs: A Mechanistic Interpretability Perspective

85% match2025

Cite this paper

@article{cyberey2025unsupervised,
  title  = {Unsupervised Concept Vector Extraction for Bias Control in LLMs},
  author = {Hannah Cyberey and Yangfeng Ji and David Evans},
  year   = {2025},
  eprint = {2502.19721},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2502.19721v3}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2502.19721.svg)](https://codeofpaper.com/paper/2502.19721)

Discussion