A Formal Framework for Understanding Length Generalization in Transformers

Xinting Huang, Andy Yang, S. Bhattamishra, Yash Sarrof, Andreas Krebs +3 more
2/12/2026
Semantic Scholar

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. Any repos shown below are weak matches — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Cite this paper

@article{huang2026formal,
  title  = {A Formal Framework for Understanding Length Generalization in Transformers},
  author = {Xinting Huang and Andy Yang and S. Bhattamishra and Yash Sarrof and Andreas Krebs and Hattie Zhou and Preetum Nakkiran and Michael Hahn},
  year   = {2026},
  url    = {https://api.semanticscholar.org/CorpusID:ef8d19af9ad288537869562cb26bf2b72dec044c},
  journal = {ICLR 2025 2025}
}

Discussion