SAKE: Steering Activations for Knowledge Editing

Marco Scialanga, Thibault Laugel, Vincent Grari, Marcin Detyniecki
3/3/2025
cs.AIcs.CLcs.LG

No official code found. Best implementation: axa-rev-research/knowledge-editing (9 stars, MIT, last commit 2025-07-09).

Abstract

As Large Langue Models have been shown to memorize real-world facts, the need to update this knowledge in a controlled and efficient manner arises. Designed with these constraints in mind, Knowledge Editing (KE) approaches propose to alter specific facts in pretrained models. However, they have been shown to suffer from several limitations, including their lack of contextual robustness and their failure to generalize to logical implications related to the fact. To overcome these issues, we propose SAKE, a steering activation method that models a fact to be edited as a distribution rather than a single prompt. Leveraging Optimal Transport, SAKE alters the LLM behavior over a whole fact-related distribution, defined as paraphrases and logical implications. Several numerical experiments demonstrate the effectiveness of this method: SAKE is thus able to perform more robust edits than its existing counterparts.

View on arXivView PDF

Code Implementations(1)

axa-rev-research/knowledge-editingCommunity69%
MIT

Code for the SAKE algorithm, described in https://arxiv.org/abs/2503.01751

91PythonMay 19, 20251 years agoMIT

Similar Papers

Persuasion Tokens for Editing Factual Knowledge in LLMs

86% match2026

REACT: Representation Extraction And Controllable Tuning to Overcome Overfitting in LLM Knowledge Editing

85% match2025

A Dual-Axis Taxonomy of Knowledge Editing for LLMs: From Mechanisms to Functions

84% match2025

EVEDIT: Event-based Knowledge Editing with Deductive Editing Boundaries

84% match2024

Joint Knowledge Editing for Information Enrichment and Probability Promotion

84% match1 repo12024

A Comprehensive Study of Knowledge Editing for Large Language Models

84% match2 repos2.9k2024

Cite this paper

@article{scialanga2025sake,
  title  = {SAKE: Steering Activations for Knowledge Editing},
  author = {Marco Scialanga and Thibault Laugel and Vincent Grari and Marcin Detyniecki},
  year   = {2025},
  eprint = {2503.01751},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2503.01751v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2503.01751.svg)](https://codeofpaper.com/paper/2503.01751)

Discussion