What Breaks in Multi-Image Schema Application? Factorial Oracle Interventions Reveal an Instance-Information Bottleneck

Yiliu Xu, Xinjin Li, Yu Ma
10/14/2026
cs.CV

No confident code implementation found for this paper.

Abstract

Modern vision-language models can accept multi-image contexts, but aggregate accuracy rarely reveals where a multi-image reasoning pipeline fails. We study a controlled schema-application task in which a model must locate two target characters in a 16-image sequence, identify their visual attributes, consult an item-specific visual codebook, and compose two ordered nonce code names. We introduce a within-item 2-by-2 factorial Oracle design: Oracle-A supplies a verified textual schema, Oracle-B supplies a verified ordered target-appearance representation, and both interventions retain the original images and final question. Across 72 new items and 1,188 analyzed model responses, Oracle-B increased strict exact accuracy by 54.6 percentage points (95% confidence interval: 46.3 to 63.0) and outperformed Oracle-A by 60.6 points (Holm-adjusted p-value no greater than .0002). An auxiliary retrieval diagnostic reached only 49.1% ordered-appearance accuracy even after removing code-name generation. Providing both Oracle sources yielded 100% accuracy across all models. These results localize the dominant recoverable deficit to upstream ordered instance recovery rather than primarily to downstream schema use and composition.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. Any repos shown below are weak matches — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

OMIBench: Benchmarking Olympiad-Level Multi-Image Reasoning in Large Vision-Language Model

86% match1 repo2026

OCR-MetaReasoning Benchmark: Evaluating the Meta-Reasoning Ability of MLLMs in Text-Rich Image Understanding

86% match1 repo12026

Do Vision-Language Models See or Guess? Measuring and Reducing Textual-Prior Reliance with a Phrasing-Controlled Benchmark

85% match2026

Meteor: Mamba-based Traversal of Rationale for Large Language and Vision Models

85% match1 repo1172024

Meteor: Mamba-based Traversal of Rationale for Large Language and Vision Models

85% match1 repo1172026

Do Vision-Language Models Truly Perform Vision Reasoning? A Rigorous Study of the Modality Gap

85% match2 repos52026

Cite this paper

@article{xu2026what,
  title  = {What Breaks in Multi-Image Schema Application? Factorial Oracle Interventions Reveal an Instance-Information Bottleneck},
  author = {Yiliu Xu and Xinjin Li and Yu Ma},
  year   = {2026},
  eprint = {2610.19611},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2610.19611v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2610.19611.svg)](https://codeofpaper.com/paper/2610.19611)

Discussion