SIGIR 2025 -- LiveRAG Challenge Report

David Carmel, Simone Filice, Guy Horowitz, Yoelle Maarek, Oren Somekh +4 more
7/7/2025
cs.CLcs.IR

No confident code implementation found for this paper.

Abstract

The LiveRAG Challenge at SIGIR 2025, held between March and May 2025, provided a competitive platform for advancing Retrieval-Augmented Generation (RAG) technologies. Participants from academia and industry were invited to develop a RAG-based question-answering system using a fixed corpus (Fineweb-10BT) and a common open-source LLM (Falcon3-10B-Instruct). The goal was to facilitate challenging comparisons of retrieval and prompting strategies. During the Live Challenge Day, 70 teams from 27 different countries provided answers and supportive information to 500 unseen questions within a strict two-hour time window. Evaluation was conducted in two stages: first an automated LLM-as-a-judge approach was used to compute correctness and faithfulness score, then a manual review of top ranked submissions was conducted. The finalists were announced on June 12, 2025, with prizes awarded during the LiveRAG Workshop at SIGIR 2025 in Padua, Italy.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. Any repos shown below are weak matches — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

RAGtifier: Evaluating RAG Generation Approaches of State-of-the-Art RAG Systems for the SIGIR LiveRAG Competition

87% match2025

LiveRAG: A diverse Q&A dataset with varying difficulty level for RAG evaluation

85% match2025

TopClustRAG at SIGIR 2025 LiveRAG Challenge

83% match2025

Evaluating Hybrid Retrieval Augmented Generation using Dynamic Test Sets: LiveRAG Challenge

83% match2025

CRAG - Comprehensive RAG Benchmark

82% match1 repo3112026

MARAGS: A Multi-Adapter System for Multi-Task Retrieval Augmented Generation Question Answering

81% match2024

Cite this paper

@article{carmel2025sigir,
  title  = {SIGIR 2025 -- LiveRAG Challenge Report},
  author = {David Carmel and Simone Filice and Guy Horowitz and Yoelle Maarek and Oren Somekh and Ran Tavory and Mehdi Ghissassi and Edo Liberty and Roy Miara},
  year   = {2025},
  eprint = {2507.04942},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2507.04942v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2507.04942.svg)](https://codeofpaper.com/paper/2507.04942)

Discussion