Quality over Quantity in Attention Layers: When Adding More Heads Hurts

Noah Amsel, Gilad Yehudai, Joan Bruna
2/12/2026

No confident code implementation found for this paper.

Semantic Scholar

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. Any repos shown below are weak matches — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Label-Focused Inductive Bias over Latent Object Features in Visual Classification

96% match2026

MIDGArD: Modular Interpretable Diffusion over Graphs for Articulated Designs

90% match2 repos42026

DynaMind: Reasoning over Abstract Video Dynamics for Embodied Decision-Making

90% match2026

MIND over Body: Adaptive Thinking using Dynamic Computation

87% match2026

Generalization Principles for Inference over Text-Attributed Graphs with Large Language Models

87% match2026

All-Purpose Mean Estimation over R: Optimal Sub-Gaussianity with Outlier Robustness and Low Moments Performance

86% match2026

Cite this paper

@article{amsel2026quality,
  title  = {Quality over Quantity in Attention Layers: When Adding More Heads Hurts},
  author = {Noah Amsel and Gilad Yehudai and Joan Bruna},
  year   = {2026},
  url    = {https://api.semanticscholar.org/CorpusID:41d764704776dec3b972d2143929370646357bd3},
  journal = {ICLR 2025 2025}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/41d764704776dec3b972d2143929370646357bd3.svg)](https://codeofpaper.com/paper/41d764704776dec3b972d2143929370646357bd3)

Discussion