Evolution of SAE Features Across Layers in LLMs

Daniel Balcells, Benjamin Lerner, Michael Oesterle, Ediz Ucar, Stefan Heimersheim
10/11/2024
cs.LG

No confident code implementation found for this paper. 2 possible matches can be shown below.

Abstract

Sparse Autoencoders for transformer-based language models are typically defined independently per layer. In this work we analyze statistical relationships between features in adjacent layers to understand how features evolve through a forward pass. We provide a graph visualization interface for features and their most similar next-layer neighbors (https://stefanhex.com/spar-2024/feature-browser/), and build communities of related features across layers. We find that a considerable amount of features are passed through from a previous layer, some features can be expressed as quasi-boolean combinations of previous features, and some features become more specialized in later layers.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. 2 weaker matches are hidden by default — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Tracking the Feature Dynamics in LLM Training: A Mechanistic Study

86% match1 repo82024

Residual Stream Analysis with Multi-Layer SAEs

85% match1 repo302024

Sparse Autoencoders Can Capture Language-Specific Concepts Across Diverse Languages

84% match1 repo12025

Unveiling Language-Specific Features in Large Language Models via Sparse Autoencoders

83% match1 repo82025

Layer-Wise Evolution of Representations in Fine-Tuned Transformers: Insights from Sparse AutoEncoders

83% match2025

Semantic Convergence: Investigating Shared Representations Across Scaled LLMs

82% match2025

Cite this paper

@article{balcells2024evolution,
  title  = {Evolution of SAE Features Across Layers in LLMs},
  author = {Daniel Balcells and Benjamin Lerner and Michael Oesterle and Ediz Ucar and Stefan Heimersheim},
  year   = {2024},
  eprint = {2410.08869},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2410.08869v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2410.08869.svg)](https://codeofpaper.com/paper/2410.08869)

Discussion