Retrieval-Augmented Perception: High-Resolution Image Perception Meets Visual RAG

Wenbin Wang, Yongcheng Jing, Liang Ding, Yingjie Wang, Li Shen +3 more
2/14/2026

No official code found. Best implementation: wl-zhao/VPD (540 stars, MIT, last commit 2023-12-21). 1 more below.

Abstract

High-resolution (HR) image perception remains a key challenge in multimodal large language models (MLLMs). To overcome the limitations of existing methods, this paper shifts away from prior dedicated heuristic approaches and revisits the most fundamental idea to HR perception by enhancing the long-context capability of MLLMs, driven by recent advances in long-context techniques like retrieval-augmented generation (RAG) for general LLMs. Towards this end, this paper presents the first study exploring the use of RAG to address HR perception challenges. Specifically, we propose Retrieval-Augmented Perception (RAP), a training-free framework that retrieves and fuses relevant image crops while preserving spatial context using the proposed Spatial-Awareness Layout. To accommodate different tasks, the proposed Retrieved-Exploration Search (RE-Search) dynamically selects the optimal number of crops based on model confidence and retrieval scores. Experimental results on HR benchmarks demonstrate the significant effectiveness of RAP, with LLaVA-v1.5-13B achieving a 43% improvement on $V^*$ Bench and 19% on HR-Bench.

View on arXivView PDF

Code Implementations(2)

wl-zhao/VPDCommunity100%
MIT

[ICCV 2023] VPD is a framework that leverages the high-level and low-level knowledge of a pre-trained text-to-image diffusion model to downstream visual perception tasks.

54033Shell, PythonMar 3, 20232 years agoMIT
DreamMr/RAPCommunity100%

Code for Retrieval-Augmented Perception （ICML 2025)

765Shell, PythonFeb 7, 20256 months ago

Similar Papers

MRD: Multi-resolution Retrieval-Detection Fusion for High-Resolution Image Understanding

88% match2025

FILA: Fine-Grained Vision Language Models

87% match2024

Divide, Conquer and Combine: A Training-Free Framework for High-Resolution Image Perception in Multimodal Large Language Models

86% match1 repo542024

RAVEN: Multitask Retrieval Augmented Vision-Language Learning

86% match2024

Enhanced Multimodal RAG-LLM for Accurate Visual Question Answering

86% match2024

Feast Your Eyes: Mixture-of-Resolution Adaptation for Multimodal Large Language Models

86% match1 repo2492026

Cite this paper

@article{wang2026retrievalaugmented,
  title  = {Retrieval-Augmented Perception: High-Resolution Image Perception Meets Visual RAG},
  author = {Wenbin Wang and Yongcheng Jing and Liang Ding and Yingjie Wang and Li Shen and Yong Luo and Bo Du and D. Tao},
  year   = {2026},
  eprint = {2503.01222},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/2503.01222},
  journal = {ICML 2025 2025}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2503.01222.svg)](https://codeofpaper.com/paper/2503.01222)

Discussion