Retrieval Augmented Generation or Long-Context LLMs? A Comprehensive Study and Hybrid Approach

Zhuowan Li, Cheng Li, Mingyang Zhang, Qiaozhu Mei, Michael Bendersky
7/23/2024
cs.CLcs.AIcs.LG

No confident code implementation found for this paper. 10 possible matches can be shown below.

Abstract

Retrieval Augmented Generation (RAG) has been a powerful tool for Large Language Models (LLMs) to efficiently process overly lengthy contexts. However, recent LLMs like Gemini-1.5 and GPT-4 show exceptional capabilities to understand long contexts directly. We conduct a comprehensive comparison between RAG and long-context (LC) LLMs, aiming to leverage the strengths of both. We benchmark RAG and LC across various public datasets using three latest LLMs. Results reveal that when resourced sufficiently, LC consistently outperforms RAG in terms of average performance. However, RAG's significantly lower cost remains a distinct advantage. Based on this observation, we propose Self-Route, a simple yet effective method that routes queries to RAG or LC based on model self-reflection. Self-Route significantly reduces the computation cost while maintaining a comparable performance to LC. Our findings provide a guideline for long-context applications of LLMs using RAG and LC.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. 10 weaker matches are hidden by default — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Long Context RAG Performance of Large Language Models

92% match2024

Long Context vs. RAG for LLMs: An Evaluation and Revisits

91% match1 repo112024

Retrieval meets Long Context Large Language Models

90% match2026

Long-Context LLMs Meet RAG: Overcoming Challenges for Long Inputs in RAG

90% match2026

Tuning LLMs by RAG Principles: Towards LLM-native Memory

89% match1 repo402025

In Defense of RAG in the Era of Long-Context Language Models

89% match2024

Cite this paper

@article{li2024retrieval,
  title  = {Retrieval Augmented Generation or Long-Context LLMs? A Comprehensive Study and Hybrid Approach},
  author = {Zhuowan Li and Cheng Li and Mingyang Zhang and Qiaozhu Mei and Michael Bendersky},
  year   = {2024},
  eprint = {2407.16833},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2407.16833v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2407.16833.svg)](https://codeofpaper.com/paper/2407.16833)

Discussion