InstUPR : Instruction-based Unsupervised Passage Reranking with Large Language Models

Chao-Wei Huang, Yun-Nung Chen
3/25/2024
cs.CLcs.IR

Official code: MiuLab/InstUPR (3 stars, last commit 2024-04-15).

Abstract

This paper introduces InstUPR, an unsupervised passage reranking method based on large language models (LLMs). Different from existing approaches that rely on extensive training with query-document pairs or retrieval-specific instructions, our method leverages the instruction-following capabilities of instruction-tuned LLMs for passage reranking without any additional fine-tuning. To achieve this, we introduce a soft score aggregation technique and employ pairwise reranking for unsupervised passage reranking. Experiments on the BEIR benchmark demonstrate that InstUPR outperforms unsupervised baselines as well as an instruction-tuned reranker, highlighting its effectiveness and superiority. Source code to reproduce all experiments is open-sourced at https://github.com/MiuLab/InstUPR

View on arXivView PDF

Code Implementations(1)

MiuLab/InstUPROfficial100%

Source code of our paper "InstUPR: Instruction-based Unsupervised Passage Reranking with Large Language Models"

31Shell, PythonMar 25, 20242 years ago
large-language-modelsllmpassage-rerankingreranking

Similar Papers

Leveraging Passage Embeddings for Efficient Listwise Reranking with Large Language Models

88% match1 repo512024

Re-Ranking Step by Step: Investigating Pre-Filtering for Re-Ranking with Large Language Models

85% match2024

Passage-specific Prompt Tuning for Passage Reranking in Question Answering with Large Language Models

85% match1 repo2024

Rank-K: Test-Time Reasoning for Listwise Reranking

84% match1 repo162025

Self-Calibrated Listwise Reranking with Large Language Models

83% match2024

LLM4Ranking: An Easy-to-use Framework of Utilizing Large Language Models for Document Reranking

83% match1 repo752025

Cite this paper

@article{huang2024instupr,
  title  = {InstUPR : Instruction-based Unsupervised Passage Reranking with Large Language Models},
  author = {Chao-Wei Huang and Yun-Nung Chen},
  year   = {2024},
  eprint = {2403.16435},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2403.16435v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2403.16435.svg)](https://codeofpaper.com/paper/2403.16435)

Discussion