Q-RAG: Long Context Multi-step Retrieval via Value-based Embedder Training

Artyom Sorokin, Nazar Buzun, Alexander Anokhin, Oleg Inozemcev, Egor Vedernikov +5 more
11/10/2025
cs.LGcs.IR

No official code found. Best implementation: griver/Q-RAG (62 stars).

Abstract

Retrieval-Augmented Generation (RAG) methods enhance LLM performance by efficiently filtering relevant context for LLMs, reducing hallucinations and inference cost. However, most existing RAG methods focus on single-step retrieval, which is often insufficient for answering complex questions that require multi-step search. Recently, multi-step retrieval approaches have emerged, typically involving the fine-tuning of small LLMs to perform multi-step retrieval. This type of fine-tuning is highly resource-intensive and does not enable the use of larger LLMs. In this work, we propose Q-RAG, a novel approach that fine-tunes the Embedder model for multi-step retrieval using reinforcement learning (RL). Q-RAG offers a competitive, resource-efficient alternative to existing multi-step retrieval methods for open-domain question answering and achieves state-of-the-art results on the popular long-context benchmarks Babilong and RULER for contexts up to 10M tokens.

View on arXivView PDF

Code Implementations(1)

griver/Q-RAGCommunity37%

Official repository for the ICLR 2026 Oral PaperšŸ”„ ā€œQ-RAG: Long Context Multi-Step Retrieval via Value-Based Embedder Trainingā€

6211

Similar Papers

Long-Context LLMs Meet RAG: Overcoming Challenges for Long Inputs in RAG

87% match2024

RAG-R1: Incentivizing the Search and Reasoning Capabilities of LLMs through Multi-query Parallelism

86% match2025

Optimizing Retrieval for RAG via Reinforcement Learning

85% match2025

Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning

85% match1 repo5.2k2025

FB-RAG: Improving RAG with Forward and Backward Lookup

85% match2025

From Sufficiency to Reflection: Reinforcement-Guided Thinking Quality in Retrieval-Augmented Reasoning for LLMs

85% match1 repo72025

Cite this paper

@article{sorokin2025qrag,
  title  = {Q-RAG: Long Context Multi-step Retrieval via Value-based Embedder Training},
  author = {Artyom Sorokin and Nazar Buzun and Alexander Anokhin and Oleg Inozemcev and Egor Vedernikov and Petr Anokhin and Mikhail Burtsev and Trushkov Alexey and Yin Wenshuai and Evgeny Burnaev},
  year   = {2025},
  eprint = {2511.07328},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2511.07328v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2511.07328.svg)](https://codeofpaper.com/paper/2511.07328)

Discussion