Between the Commits: Process, Error, and Claim Reliability in a Wholly AI-Authored Codebase

Douglas Leith
9/24/2026
cs.SEcs.AI

No confident code implementation found for this paper.

Abstract

We present: (i) a new dataset consisting of the full development history of a 21,000-line Python tool built entirely by Claude AI, with no human-authored code or tests, (ii) two code-provenance tracing tools, (iii) three taxonomies for instruction intent, commit provenance, and response reliability, (iv) application of these to analyse the dataset. We find that: (i) user coding agent CLI instructions differ in kind from IDE-chat instructions, with a greater focus on comprehension, planning and consultation, (ii) code development is mainly proactive, (iii) 14.3% of AI code-generation events contain a real error later caught by the AI-authored test suite, (iv) roughly 1 in 4-5 of the AI's interactive responses contains one or more factual errors.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. Any repos shown below are weak matches — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

SWE-chat: Coding Agent Interactions From Real Users in the Wild

86% match1 repo332026

From Tokens to Files: Multi-Granularity Detection and Localization of AI-Involved Code

85% match2026

TraceCoder: Explainable and Auditable Code Generation with Position-Key Snippet Versioning

84% match2026

SPECMINE: A Large-Scale Corpus of Spec-Driven Development Artifacts

84% match2026

Investigating Autonomous Agent Contributions in the Wild: Activity Patterns and Code Change over Time

83% match2026

Cracks in The Stack: Hidden Vulnerabilities and Licensing Risks in LLM Pre-Training Datasets

83% match2025

Cite this paper

@article{leith2026between,
  title  = {Between the Commits: Process, Error, and Claim Reliability in a Wholly AI-Authored Codebase},
  author = {Douglas Leith},
  year   = {2026},
  eprint = {2609.29744},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2609.29744v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2609.29744.svg)](https://codeofpaper.com/paper/2609.29744)

Discussion