Planning Ahead in Generative Retrieval: Guiding Autoregressive Generation through Simultaneous Decoding

Hansi Zeng, Chen Luo, Hamed Zamani
4/22/2024
cs.IRcs.CL

No official code found. Best implementation: HansiZeng/PAG (32 stars, last commit 2024-04-24).

Abstract

This paper introduces PAG-a novel optimization and decoding approach that guides autoregressive generation of document identifiers in generative retrieval models through simultaneous decoding. To this aim, PAG constructs a set-based and sequential identifier for each document. Motivated by the bag-of-words assumption in information retrieval, the set-based identifier is built on lexical tokens. The sequential identifier, on the other hand, is obtained via quantizing relevance-based representations of documents. Extensive experiments on MSMARCO and TREC Deep Learning Track data reveal that PAG outperforms the state-of-the-art generative retrieval model by a large margin (e.g., 15.6% MRR improvements on MS MARCO), while achieving 22x speed up in terms of query latency.

View on arXivView PDF

Code Implementations(1)

HansiZeng/PAGCommunity100%

[SIGIR 2024] The official repo for paper "Planning Ahead in Generative Retrieval: Guiding Autoregressive Generation through Simultaneous Decoding"

326Shell, PythonJan 25, 20242 years ago

Similar Papers

Scaling the Vocabulary of Non-autoregressive Models for Efficient Generative Retrieval

84% match2024

Exploring Training and Inference Scaling Laws in Generative Retrieval

82% match1 repo42025

Learning to Rank in Generative Retrieval

82% match1 repo212023

GLEN: Generative Retrieval via Lexical Index Learning

82% match1 repo292023

Autoregressive Ranking: Bridging the Gap Between Dual and Cross Encoders

82% match2026

Learning to Tokenize for Generative Retrieval

81% match1 repo862023

Cite this paper

@article{zeng2024planning,
  title  = {Planning Ahead in Generative Retrieval: Guiding Autoregressive Generation through Simultaneous Decoding},
  author = {Hansi Zeng and Chen Luo and Hamed Zamani},
  year   = {2024},
  eprint = {2404.14600},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2404.14600v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2404.14600.svg)](https://codeofpaper.com/paper/2404.14600)

Discussion