RbFT: Robust Fine-tuning for Retrieval-Augmented Generation against Retrieval Defects

Yiteng Tu, Weihang Su, Yujia Zhou, Yiqun Liu, Qingyao Ai
1/30/2025
cs.CLcs.IR

No official code found. Best implementation: StibiumT16/Robust-Fine-tuning (19 stars, last commit 2025-01-31).

Abstract

Retrieval-augmented generation (RAG) enhances large language models (LLMs) by integrating external knowledge retrieved from a knowledge base. However, its effectiveness is fundamentally constrained by the reliability of both the retriever and the knowledge base. In real-world scenarios, imperfections in these components often lead to the retrieval of noisy, irrelevant, or misleading counterfactual information, ultimately undermining the trustworthiness of RAG systems. To address this challenge, we propose Robust Fine-Tuning (RbFT), a method designed to enhance the resilience of LLMs against retrieval defects through two targeted fine-tuning tasks. Experimental results demonstrate that RbFT significantly improves the robustness of RAG systems across diverse retrieval conditions, surpassing existing methods while maintaining high inference efficiency and compatibility with other robustness techniques.

View on arXivView PDF

Code Implementations(1)

StibiumT16/Robust-Fine-tuningCommunity100%

Code for Robust Fine-tuning (RbFT)

192Shell, PythonJan 16, 20251 years ago

Similar Papers

Finetune-RAG: Fine-Tuning Language Models to Resist Hallucination in Retrieval-Augmented Generation

90% match1 repo472025

Astute RAG: Overcoming Imperfect Retrieval Augmentation and Knowledge Conflicts for Large Language Models

89% match2024

Investigating the Robustness of Retrieval-Augmented Generation at the Query Level

87% match2025

After Retrieval, Before Generation: Enhancing the Trustworthiness of Large Language Models in Retrieval-Augmented Generation

87% match2025

Trustworthiness in Retrieval-Augmented Generation Systems: A Survey

87% match1 repo162024

Systematic Knowledge Injection into Large Language Models via Diverse Augmentation for Domain-Specific RAG

87% match1 repo22025

Cite this paper

@article{tu2025rbft,
  title  = {RbFT: Robust Fine-tuning for Retrieval-Augmented Generation against Retrieval Defects},
  author = {Yiteng Tu and Weihang Su and Yujia Zhou and Yiqun Liu and Qingyao Ai},
  year   = {2025},
  eprint = {2501.18365},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2501.18365v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2501.18365.svg)](https://codeofpaper.com/paper/2501.18365)

Discussion