Can sparse autoencoders be used to decompose and interpret steering vectors?

Harry Mayne, Yushi Yang, Adam Mahdi
11/13/2024
cs.LGcs.AIcs.CL

No official code found. Best implementation: HarryMayne/SV_interpretability (8 stars, last commit 2024-11-12).

Abstract

Steering vectors are a promising approach to control the behaviour of large language models. However, their underlying mechanisms remain poorly understood. While sparse autoencoders (SAEs) may offer a potential method to interpret steering vectors, recent findings show that SAE-reconstructed vectors often lack the steering properties of the original vectors. This paper investigates why directly applying SAEs to steering vectors yields misleading decompositions, identifying two reasons: (1) steering vectors fall outside the input distribution for which SAEs are designed, and (2) steering vectors can have meaningful negative projections in feature directions, which SAEs are not designed to accommodate. These limitations hinder the direct use of SAEs for interpreting steering vectors.

View on arXivView PDF

Code Implementations(1)

HarryMayne/SV_interpretabilityCommunity100%

Code for the paper "Can sparse autoencoders be used to decompose and interpret steering vectors?"

80Nov 12, 20241 years ago

Similar Papers

Improving Steering Vectors by Targeting Sparse Autoencoder Features

91% match1 repo302024

Enhancing LLM Steering through Sparse Autoencoder-Based Vector Refinement

90% match2025

SAEs Are Good for Steering -- If You Select the Right Features

88% match1 repo102025

A Comparative Analysis of Sparse Autoencoder and Activation Difference in Language Model Steering

86% match2025

Identifiable Steering via Sparse Autoencoding of Multi-Concept Shifts

85% match2025

Does higher interpretability imply better utility? A Pairwise Analysis on Sparse Autoencoders

84% match2025

Cite this paper

@article{mayne2024sparse,
  title  = {Can sparse autoencoders be used to decompose and interpret steering vectors?},
  author = {Harry Mayne and Yushi Yang and Adam Mahdi},
  year   = {2024},
  eprint = {2411.08790},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2411.08790v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2411.08790.svg)](https://codeofpaper.com/paper/2411.08790)

Discussion