How to use and interpret activation patching

Stefan Heimersheim, Neel Nanda
4/23/2024
cs.LG

No confident code implementation found for this paper.

Abstract

Activation patching is a popular mechanistic interpretability technique, but has many subtleties regarding how it is applied and how one may interpret the results. We provide a summary of advice and best practices, based on our experience using this technique in practice. We include an overview of the different ways to apply activation patching and a discussion on how to interpret the results. We focus on what evidence patching experiments provide about circuits, and on the choice of metric and associated pitfalls.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. Any repos shown below are weak matches — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Towards Best Practices of Activation Patching in Language Models: Metrics and Methods

86% match2026

AtP*: An efficient and scalable method for localizing LLM behaviour to components

78% match2024

Is This the Subspace You Are Looking for? An Interpretability Illusion for Subspace Activation Patching

77% match1 repo102026

Circuit Insights: Towards Interpretability Beyond Activations

77% match2025

Investigating task-specific prompts and sparse autoencoders for activation monitoring

75% match2025

RelP: Faithful and Efficient Circuit Discovery in Language Models via Relevance Patching

75% match1 repo292025

Cite this paper

@article{heimersheim2024interpret,
  title  = {How to use and interpret activation patching},
  author = {Stefan Heimersheim and Neel Nanda},
  year   = {2024},
  eprint = {2404.15255},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2404.15255v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2404.15255.svg)](https://codeofpaper.com/paper/2404.15255)

Discussion