What Is the Minimum Architecture for Prolepsis? Early Irrevocable Commitment Across Tasks in Small Transformers

Éric Jacopin
4/16/2026
cs.LGcs.AIcs.CL

No confident code implementation found for this paper.

Abstract

When do transformers commit to a decision, and what prevents them from correcting it? We introduce \textbf{prolepsis}: a transformer commits early, task-specific attention heads sustain the commitment, and no layer corrects it. Replicating \citeauthor{lindsey2025biology}'s (\citeyear{lindsey2025biology}) planning-site finding on open models (Gemma~2 2B, Llama~3.2 1B), we ask five questions. (Q1)~Planning is invisible to six residual-stream methods; CLTs are necessary. (Q2)~The planning-site spike replicates with identical geometry. (Q3)~Specific attention heads route the decision to the output, filling a gap flagged as invisible to attribution graphs. (Q4)~Search requires ${\leq}16$ layers; commitment requires more. (Q5)~Factual recall shows the same motif at a different network depth, with zero overlap between recurring planning heads and the factual top-10. Prolepsis is architectural: the template is shared, the routing substrates differ. All experiments run on a single consumer GPU (16\,GB VRAM).

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. Any repos shown below are weak matches — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Through the Looking Glass: Directly Reading and Writing Transformers

83% match2026

Learning to Compute Is Not Learning to Retain or Reuse in Looped Transformers

81% match2026

Transformers Stop Thinking Too Early, and a Tiny LoRA Fixes It

80% match1 repo32026

Routing Probes Can Improve Without New Information: An Exact-Null Audit of Uncertainty Beyond Model Outputs

80% match2026

What Does Layer-Importance Reveal About Transformers and State-Space Models?

80% match2026

Contrastive Projection: Reading Transformer Internals by Differencing Logit Lenses

80% match2026

Cite this paper

@article{jacopin2026what,
  title  = {What Is the Minimum Architecture for Prolepsis? Early Irrevocable Commitment Across Tasks in Small Transformers},
  author = {Éric Jacopin},
  year   = {2026},
  eprint = {2604.15010},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2604.15010v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2604.15010.svg)](https://codeofpaper.com/paper/2604.15010)

Discussion