Does Liking Yellow Imply Driving a School Bus? Semantic Leakage in Language Models

Hila Gonen, Terra Blevins, Alisa Liu, Luke Zettlemoyer, Noah A. Smith
8/12/2024
cs.CL

No confident code implementation found for this paper. 1 possible match can be shown below.

Abstract

Despite their wide adoption, the biases and unintended behaviors of language models remain poorly understood. In this paper, we identify and characterize a phenomenon never discussed before, which we call semantic leakage, where models leak irrelevant information from the prompt into the generation in unexpected ways. We propose an evaluation setting to detect semantic leakage both by humans and automatically, curate a diverse test suite for diagnosing this behavior, and measure significant semantic leakage in 13 flagship models. We also show that models exhibit semantic leakage in languages besides English and across different settings and generation scenarios. This discovery highlights yet another type of bias in language models that affects their generation patterns and behavior.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. 1 weaker match is hidden by default — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Am I Blue or Is My Hobby Counting Teardrops? Expression Leakage in Large Language Models as a Symptom of Irrelevancy Disruption

86% match2025

Semantic and Structural Analysis of Implicit Biases in Large Language Models: An Interpretable Approach

82% match2025

From n-gram to Attention: How Model Architectures Learn and Propagate Bias in Language Modeling

82% match2025

In Machina N400: Pinpointing Where a Causal Language Model Detects Semantic Violations

82% match2025

Can Language Model Understand Word Semantics as A Chatbot? An Empirical Study of Language Model Internal External Mismatch

82% match2024

modeLing: A Novel Dataset for Testing Linguistic Reasoning in Language Models

81% match2024

Cite this paper

@article{gonen2024does,
  title  = {Does Liking Yellow Imply Driving a School Bus? Semantic Leakage in Language Models},
  author = {Hila Gonen and Terra Blevins and Alisa Liu and Luke Zettlemoyer and Noah A. Smith},
  year   = {2024},
  eprint = {2408.06518},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2408.06518v3}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2408.06518.svg)](https://codeofpaper.com/paper/2408.06518)

Discussion