RepBERT: Contextualized Text Embeddings for First-Stage Retrieval

Jingtao Zhan, Jiaxin Mao, Yiqun Liu, Min Zhang, Shaoping Ma
6/28/2020
cs.IR

Official code: jingtaozhan/RepBERT-Index (66 stars, BSD-3-Clause, PyTorch, last commit 2021-10-13).

Abstract

Although exact term match between queries and documents is the dominant method to perform first-stage retrieval, we propose a different approach, called RepBERT, to represent documents and queries with fixed-length contextualized embeddings. The inner products of query and document embeddings are regarded as relevance scores. On MS MARCO Passage Ranking task, RepBERT achieves state-of-the-art results among all initial retrieval techniques. And its efficiency is comparable to bag-of-words methods.

View on arXivView PDF

Code Implementations(1)

jingtaozhan/RepBERT-IndexOfficial100%
PyTorchBSD-3-Clause

RepBERT is a competitive first-stage retrieval technique. It represents documents and queries with fixed-length contextualized embeddings. The inner products of them are regarded as relevance scores. Its efficiency is comparable to bag-of-words methods.

6610PythonJun 23, 20205 years agoBSD-3-Clause
information-retrievalpytorchweb-search

Similar Papers

SPM: Structured Pretraining and Matching Architectures for Relevance Modeling in Meituan Search

82% match2023

GAR-meets-RAG Paradigm for Zero-Shot Information Retrieval

81% match2023

Neural Vector Spaces for Unsupervised Information Retrieval

80% match1 repo492017

Topic-Grained Text Representation-based Model for Document Retrieval

80% match2022

Context-Aware Sentence/Passage Term Importance Estimation For First Stage Retrieval

80% match1 repo3262019

Lexically-Accelerated Dense Retrieval

79% match1 repo72023

Cite this paper

@article{zhan2020repbert,
  title  = {RepBERT: Contextualized Text Embeddings for First-Stage Retrieval},
  author = {Jingtao Zhan and Jiaxin Mao and Yiqun Liu and Min Zhang and Shaoping Ma},
  year   = {2020},
  eprint = {2006.15498},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/2006.15498}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2006.15498.svg)](https://codeofpaper.com/paper/2006.15498)

Discussion