Just Enough Shifts: Mitigating Over-Refusal in Aligned Language Models with Targeted Representation Fine-Tuning

Mahavir Dabas, Si Chen, Charles Fleming, Ming Jin, Ruoxi Jia
7/6/2025
cs.LGcs.AI

No official code found. Best implementation: MahavirDabas18/ACTOR (2 stars, last commit 2025-07-06).

Abstract

Safety alignment is crucial for large language models (LLMs) to resist malicious instructions but often results in over-refusals, where benign prompts are unnecessarily rejected, impairing user experience and model utility. We introduce ACTOR (Activation-Based Training for Over-Refusal Reduction), a robust and compute- and data-efficient training framework that minimizes over-refusals by leveraging internal activation patterns from diverse queries. ACTOR precisely identifies and adjusts the activation components that trigger refusals, providing stronger control over the refusal mechanism. By fine-tuning only a single model layer, ACTOR effectively reduces over-refusals across multiple benchmarks while maintaining the model's ability to handle harmful queries and preserve overall utility.

View on arXivView PDF

Code Implementations(1)

MahavirDabas18/ACTORCommunity67%

The official implementation of the ICML 2025 paper, Just Enough Shifts: Mitigating Over-Refusal in Aligned Language Models with Targeted Representation Fine-Tuning

20Jul 6, 20251 years ago

Similar Papers

OR-Bench: An Over-Refusal Benchmark for Large Language Models

87% match1 repo282024

OR-Bench: An Over-Refusal Benchmark for Large Language Models

87% match1 repo282026

Refuse Whenever You Feel Unsafe: Improving Safety in LLMs via Decoupled Refusal Training

87% match1 repo722024

Refusal Tokens: A Simple Way to Calibrate Refusals in Large Language Models

86% match1 repo112024

From Threat to Tool: Leveraging Refusal-Aware Injection Attacks for Safety Alignment

86% match2025

Refusal Steering: Fine-grained Control over LLM Refusal Behaviour for Sensitive Topics

86% match2025

Cite this paper

@article{dabas2025just,
  title  = {Just Enough Shifts: Mitigating Over-Refusal in Aligned Language Models with Targeted Representation Fine-Tuning},
  author = {Mahavir Dabas and Si Chen and Charles Fleming and Ming Jin and Ruoxi Jia},
  year   = {2025},
  eprint = {2507.04250},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2507.04250v1},
  journal = {ICML 2025 2025}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2507.04250.svg)](https://codeofpaper.com/paper/2507.04250)

Discussion