Contextualized Sparse Representations for Real-Time Open-Domain Question Answering

Jinhyuk Lee, Minjoon Seo, Hannaneh Hajishirzi, Jaewoo Kang
11/7/2019
cs.CLcs.IR

Official code: jhyuklee/sparc (49 stars, Apache-2.0, last commit 2020-10-06).

Abstract

Open-domain question answering can be formulated as a phrase retrieval problem, in which we can expect huge scalability and speed benefit but often suffer from low accuracy due to the limitation of existing phrase representation models. In this paper, we aim to improve the quality of each phrase embedding by augmenting it with a contextualized sparse representation (Sparc). Unlike previous sparse vectors that are term-frequency-based (e.g., tf-idf) or directly learned (only few thousand dimensions), we leverage rectified self-attention to indirectly learn sparse vectors in n-gram vocabulary space. By augmenting the previous phrase retrieval model (Seo et al., 2019) with Sparc, we show 4%+ improvement in CuratedTREC and SQuAD-Open. Our CuratedTREC score is even better than the best known retrieve & read model with at least 45x faster inference speed.

View on arXivView PDF

Code Implementations(1)

jhyuklee/sparcOfficial100%
Apache-2.0

ACL'2020: Contextualized Sparse Representations for Real-Time Open-Domain Question Answering

4910CSS, HTMLApr 24, 20206 years agoApache-2.0

Similar Papers

Learning Dense Representations of Phrases at Scale

91% match1 repo6062020

Dense Passage Retrieval for Open-Domain Question Answering

85% match1 repo1.9k2020

PiC: A Phrase-in-Context Dataset for Phrase Understanding and Semantic Search

82% match1 repo72022

Dense Sparse Retrieval: Using Sparse Language Models for Inference Efficient Dense Retrieval

81% match2023

Multimodal Learned Sparse Retrieval with Probabilistic Expansion Control

81% match1 repo132024

Can a Frozen Pretrained Language Model be used for Zero-shot Neural Retrieval on Entity-centric Questions?

81% match2023

Cite this paper

@article{lee2019contextualized,
  title  = {Contextualized Sparse Representations for Real-Time Open-Domain Question Answering},
  author = {Jinhyuk Lee and Minjoon Seo and Hannaneh Hajishirzi and Jaewoo Kang},
  year   = {2019},
  eprint = {1911.02896},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/1911.02896}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/1911.02896.svg)](https://codeofpaper.com/paper/1911.02896)

Discussion