No confident code implementation found for this paper.
We construct a 3-layer constant-dimension transformer, recognizing the parity language, where neither parameter matrices nor the positional encoding depend on the input length. This improves upon a construction of Chiang and Cholak who use a positional encoding, depending on the input length (but their construction has 2 layers).
No confident code match yet
We couldn't find an author-owned or strongly-evidenced community implementation for this paper. Any repos shown below are weak matches — verify before relying on them.
No code implementations found yet.
Know of an implementation? Let us know in the comments below!
Parity, Sensitivity, and Transformers
Theoretical Analysis of Positional Encodings in Transformer Models: Impact on Expressiveness and Generalization
Transformers in Uniform TC$^0$
Theoretical Analysis of Hierarchical Language Recognition and Generation by Transformers without Positional Encoding
Equivalence of Context and Parameter Updates in Modern Transformer Blocks
Transformers Can Overcome the Curse of Dimensionality: A Theoretical Study from an Approximation Perspective
@article{kozachinskiy2025completely,
title = {A completely uniform transformer for parity},
author = {Alexander Kozachinskiy and Tomasz Steifer},
year = {2025},
eprint = {2501.02535},
archivePrefix = {arXiv},
url = {http://arxiv.org/abs/2501.02535v1}
}Shows whether official code is linked for this paper.
[](https://codeofpaper.com/paper/2501.02535)