Padding to k: A Minimum Count in the Schema Makes Open-Weight Models Pad Their Extraction Lists

Abhishek Barali, Aayush Paudel
10/22/2026
cs.CL

Official code: AbhishekBarali/padding-to-k (0 stars, last commit 2026-10-22).

Abstract

Extraction pipelines ask language models for JSON that follows a schema, and the schema can set a minimum number of list entries. We ask what models do when the document holds fewer items than that minimum. Under a preregistered plan, we test eight open-weight models on 558 real documents from nine annotated datasets, with six more models on hosted services as a reference. Three findings stand out. First, models treat the count as a target: a minimum shown in the schema, but not enforced, raises the share of lists longer than the true count from 34.5% to 55.4%. Second, the padding is hard to catch, because most added entries are real text taken from the document; about half of them pass a check that each entry occurs in the source. Third, common fixes help only partly: a line saying fewer items are fine removes the effect on average but misses more true items, a count in prose ("up to k") also raises over-counting, an "N/A" option under an enforced minimum still leaves half the lists padded, and re-asking with the validation error makes padding far more common. We release the code, answer lists and run records at https://github.com/AbhishekBarali/padding-to-k

View on arXivView PDF

Code Implementations(1)

AbhishekBarali/padding-to-kOfficial100%

Code, data and results for the paper 'Padding to k: A Minimum Count in the Schema Makes Open-Weight Models Pad Their Extraction Lists'

01TeX, JinjaOct 21, 20263 days agoNOASSERTION
benchmarkinformation-extractionjson-schemallmnlp+2 more

Similar Papers

CombEval: A Framework for Evaluating Combinatorial Counting in Large Language Models

78% match1 repo2026

Replay the Curvature: Accurate and Scalable NVFP4 Quantization for Large Language Model Inference

78% match2026

Let's Sample Step by Step: Adaptive-Consistency for Efficient Reasoning and Coding with LLMs

78% match1 repo412023

One Token Away from Collapse: The Fragility of Instruction-Tuned Helpfulness

78% match2026

Honey, I Shrunk the Language: Language Model Behavior at Reduced Scale

78% match2023

The Right Answer, the Wrong Direction: Why Transformers Fail at Counting and How to Fix It

78% match2026

Cite this paper

@article{barali2026padding,
  title  = {Padding to k: A Minimum Count in the Schema Makes Open-Weight Models Pad Their Extraction Lists},
  author = {Abhishek Barali and Aayush Paudel},
  year   = {2026},
  eprint = {2610.30997},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2610.30997v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2610.30997.svg)](https://codeofpaper.com/paper/2610.30997)

Discussion