Project Aletheia: Verifier-Guided Distillation of Backtracking for Small Language Models

Aradhya Dixit, Tianxi Liang, Jai Telang
1/14/2026
cs.CL

No confident code implementation found for this paper.

Abstract

Small Language Models (SLMs, under 10B parameters) are attractive for private, on-device deployment, yet they frequently fail on strict constraint-satisfaction problems due to linear, overconfident reasoning traces that do not recover from early mistakes. We introduce Verifier-Guided Distillation, a training protocol that transfers the process of error repair - explicit conflict detection and backtracking - rather than only correct final answers. By training a 7B model on verified reasoning traces that include mistakes and self-corrections, we show that latent verification behavior can emerge in small models, enabling them to occasionally stop, detect contradictions, and revise earlier assumptions.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. Any repos shown below are weak matches — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Cite this paper

@article{dixit2026project,
  title  = {Project Aletheia: Verifier-Guided Distillation of Backtracking for Small Language Models},
  author = {Aradhya Dixit and Tianxi Liang and Jai Telang},
  year   = {2026},
  eprint = {2601.14290},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2601.14290v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2601.14290.svg)](https://codeofpaper.com/paper/2601.14290)

Discussion