Seeing is Believing: Brain-Inspired Modular Training for Mechanistic Interpretability

Ziming Liu, Eric Gan, Max Tegmark
5/4/2023
cs.NEcond-mat.dis-nncs.AI

No official code found. Best implementation: KindXiaoming/BIMT (177 stars, last commit 2023-05-19).

Abstract

We introduce Brain-Inspired Modular Training (BIMT), a method for making neural networks more modular and interpretable. Inspired by brains, BIMT embeds neurons in a geometric space and augments the loss function with a cost proportional to the length of each neuron connection. We demonstrate that BIMT discovers useful modular neural networks for many simple tasks, revealing compositional structures in symbolic formulas, interpretable decision boundaries and features for classification, and mathematical structure in algorithmic datasets. The ability to directly see modules with the naked eye can complement current mechanistic interpretability strategies such as probes, interventions or staring at all weights.

View on arXivView PDF

Code Implementations(1)

KindXiaoming/BIMTCommunity100%

Brain-Inspired Modular Training (BIMT), a method for making neural networks more modular and interpretable.

17730Jupyter NotebookMay 19, 20233 years ago

Similar Papers

Evaluating Brain-Inspired Modular Training in Automated Circuit Discovery for Mechanistic Interpretability

88% match2024

Training Neural Networks for Modularity aids Interpretability

84% match2024

A Cortically Inspired Architecture for Modular Perceptual AI

83% match2026

From Neurons to Neutrons: A Case Study in Interpretability

82% match1 repo52024

The Cognitive Revolution in Interpretability: From Explaining Behavior to Interpreting Representations and Algorithms

82% match2024

Unboxing the Black Box: Mechanistic Interpretability for Algorithmic Understanding of Neural Networks

81% match2025

Cite this paper

@article{liu2023seeing,
  title  = {Seeing is Believing: Brain-Inspired Modular Training for Mechanistic Interpretability},
  author = {Ziming Liu and Eric Gan and Max Tegmark},
  year   = {2023},
  eprint = {2305.08746},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2305.08746v3}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2305.08746.svg)](https://codeofpaper.com/paper/2305.08746)

Discussion