A Percolation Model of Emergence: Analyzing Transformers Trained on a Formal Language

E. Lubana, Kyogo Kawaguchi, Robert P. Dick, Hidenori Tanaka
2/12/2026

No official code found. Best implementation: EkdeepSLubana/ConceptPercolation (9 stars, last commit 2024-10-11).

Semantic Scholar

Code Implementations(1)

EkdeepSLubana/ConceptPercolationCommunity65%

Codebase for the paper "A Percolation Model of Emergence: Analyzing Transformers Trained on a Formal Language"

90PythonAug 21, 20242 years ago

Cite this paper

@article{lubana2026percolation,
  title  = {A Percolation Model of Emergence: Analyzing Transformers Trained on a Formal Language},
  author = {E. Lubana and Kyogo Kawaguchi and Robert P. Dick and Hidenori Tanaka},
  year   = {2026},
  url    = {https://api.semanticscholar.org/CorpusID:a21b76c344c04721825796b22786d91f3c83b56f},
  journal = {ICLR 2025 2025}
}

Discussion