RAG in the Wild: On the (In)effectiveness of LLMs with Mixture-of-Knowledge Retrieval Augmentation

Ran Xu, Yuchen Zhuang, Yue Yu, Haoyu Wang, Wenqi Shi +1 more
7/26/2025
cs.CLcs.AIcs.LG

Official code: ritaranx/RAG_in_the_Wild (1 star, MIT, last commit 2025-10-07).

Abstract

Retrieval-augmented generation (RAG) enhances large language models (LLMs) by integrating external knowledge retrieved at inference time. While RAG demonstrates strong performance on benchmarks largely derived from general-domain corpora like Wikipedia, its effectiveness under realistic, diverse retrieval scenarios remains underexplored. We evaluated RAG systems using MassiveDS, a large-scale datastore with mixture of knowledge, and identified critical limitations: retrieval mainly benefits smaller models, rerankers add minimal value, and no single retrieval source consistently excels. Moreover, current LLMs struggle to route queries across heterogeneous knowledge sources. These findings highlight the need for adaptive retrieval strategies before deploying RAG in real-world settings. Our code and data can be found at https://github.com/ritaranx/RAG_in_the_Wild.

View on arXivView PDF

Code Implementations(1)

ritaranx/RAG_in_the_WildOfficial100%
MIT
10PythonJul 13, 20251 years agoMIT

Similar Papers

Astute RAG: Overcoming Imperfect Retrieval Augmentation and Knowledge Conflicts for Large Language Models

91% match2024

When Retrieval Succeeds and Fails: Rethinking Retrieval-Augmented Generation for LLMs

90% match2025

RAG Foundry: A Framework for Enhancing LLMs for Retrieval Augmented Generation

90% match1 repo7712024

A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Models

89% match2024

RAGLAB: A Modular and Research-Oriented Unified Framework for Retrieval-Augmented Generation

89% match1 repo3142024

Systematic Knowledge Injection into Large Language Models via Diverse Augmentation for Domain-Specific RAG

89% match1 repo22025

Cite this paper

@article{xu2025wild,
  title  = {RAG in the Wild: On the (In)effectiveness of LLMs with Mixture-of-Knowledge Retrieval Augmentation},
  author = {Ran Xu and Yuchen Zhuang and Yue Yu and Haoyu Wang and Wenqi Shi and Carl Yang},
  year   = {2025},
  eprint = {2507.20059},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2507.20059v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2507.20059.svg)](https://codeofpaper.com/paper/2507.20059)

Discussion