Reinforcement Learning for Long-Horizon Multi-Turn Search Agents

Vivek Kalyan, Martin Andrews
10/28/2025
cs.CL

No confident code implementation found for this paper.

Abstract

Large Language Model (LLM) agents can leverage multiple turns and tools to solve complex tasks, with prompt-based approaches achieving strong performance. This work demonstrates that Reinforcement Learning (RL) can push capabilities significantly further by learning from experience. Through experiments on a legal document search benchmark, we show that our RL-trained 14 Billion parameter model outperforms frontier class models (85% vs 78% accuracy). In addition, we explore turn-restricted regimes, during training and at test-time, that show these agents achieve better results if allowed to operate over longer multi-turn horizons.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. Any repos shown below are weak matches — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

R1-Searcher: Incentivizing the Search Capability in LLMs via Reinforcement Learning

89% match1 repo7232025

SSRL: Self-Search Reinforcement Learning

88% match1 repo2122025

Meta-RL Induces Exploration in Language Agents

88% match1 repo452025

Guiding Exploration in Reinforcement Learning Through LLM-Augmented Observations

88% match2025

TSR: Trajectory-Search Rollouts for Multi-Turn RL of LLM Agents

88% match2026

A Practitioner's Guide to Multi-turn Agentic Reinforcement Learning

88% match1 repo832025

Cite this paper

@article{kalyan2025reinforcement,
  title  = {Reinforcement Learning for Long-Horizon Multi-Turn Search Agents},
  author = {Vivek Kalyan and Martin Andrews},
  year   = {2025},
  eprint = {2510.24126},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2510.24126v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2510.24126.svg)](https://codeofpaper.com/paper/2510.24126)

Discussion