Separate the Wheat from the Chaff: A Post-Hoc Approach to Safety Re-Alignment for Fine-Tuned Language Models

Di Wu, Xin Lu, Yanyan Zhao, Bing Qin
12/15/2024
cs.CL

No official code found. Best implementation: pikepokenew/IRR (3 stars, last commit 2025-05-28).

Abstract

Although large language models (LLMs) achieve effective safety alignment at the time of release, they still face various safety challenges. A key issue is that fine-tuning often compromises the safety alignment of LLMs. To address this issue, we propose a method named IRR (Identify, Remove, and Recalibrate for Safety Realignment) that performs safety realignment for LLMs. The core of IRR is to identify and remove unsafe delta parameters from the fine-tuned models, while recalibrating the retained ones. We evaluate the effectiveness of IRR across various datasets, including both full fine-tuning and LoRA methods. Our results demonstrate that IRR significantly enhances the safety performance of fine-tuned models on safety benchmarks, such as harmful queries and jailbreak attacks, while maintaining their performance on downstream tasks. The source code is available at: https://anonymous.4open.science/r/IRR-BD4F.

View on arXivView PDF

Code Implementations(1)

pikepokenew/IRRCommunity100%

The official code repo for "Separate the Wheat from the Chaff: A Post-Hoc Approach to Safety Re-Alignment for Fine-Tuned Language Models" in ACL 2025 findings.

30Shell, PythonDec 12, 20241 years ago

Similar Papers

Locking Down the Finetuned LLMs Safety

91% match1 repo112024

SafeMERGE: Preserving Safety Alignment in Fine-Tuned Large Language Models via Selective Layer-Wise Model Merging

90% match1 repo152025

A safety realignment framework via subspace-oriented model fusion for large language models

90% match1 repo62024

Safeguard Fine-Tuned LLMs Through Pre- and Post-Tuning Model Merging

90% match2024

Why LLM Safety Guardrails Collapse After Fine-tuning: A Similarity Analysis Between Alignment and Fine-tuning Datasets

89% match2025

Safety at One Shot: Patching Fine-Tuned LLMs with A Single Instance

89% match2026

Cite this paper

@article{wu2024separate,
  title  = {Separate the Wheat from the Chaff: A Post-Hoc Approach to Safety Re-Alignment for Fine-Tuned Language Models},
  author = {Di Wu and Xin Lu and Yanyan Zhao and Bing Qin},
  year   = {2024},
  eprint = {2412.11041},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2412.11041v3}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2412.11041.svg)](https://codeofpaper.com/paper/2412.11041)

Discussion