M3: A Multi-Task Mixed-Objective Learning Framework for Open-Domain Multi-Hop Dense Sentence Retrieval

Yang Bai, Anthony Colas, Christan Grant, Daisy Zhe Wang
3/21/2024
cs.IRcs.CLcs.LG

Official code: TonyBY/M3 (1 star, Apache-2.0, last commit 2024-12-01).

Abstract

In recent research, contrastive learning has proven to be a highly effective method for representation learning and is widely used for dense retrieval. However, we identify that relying solely on contrastive learning can lead to suboptimal retrieval performance. On the other hand, despite many retrieval datasets supporting various learning objectives beyond contrastive learning, combining them efficiently in multi-task learning scenarios can be challenging. In this paper, we introduce M3, an advanced recursive Multi-hop dense sentence retrieval system built upon a novel Multi-task Mixed-objective approach for dense text representation learning, addressing the aforementioned challenges. Our approach yields state-of-the-art performance on a large-scale open-domain fact verification benchmark dataset, FEVER. Code and data are available at: https://github.com/TonyBY/M3

View on arXivView PDF

Code Implementations(1)

TonyBY/M3Official100%
Apache-2.0

M3: A Multi-Hop Dense Sentence Retrieval System for Fact Checking

10PythonOct 17, 20231 years agoApache-2.0

Similar Papers

LexLIP: Lexicon-Bottlenecked Language-Image Pre-Training for Large-Scale Image-Text Sparse Retrieval

80% match1 repo392026

CRE-T1 Preview Technical Report: Beyond Contrastive Learning for Reasoning-Intensive Retrieval

80% match2026

OPERA: A Reinforcement Learning--Enhanced Orchestrated Planner-Executor Architecture for Reasoning-Oriented Multi-Hop Retrieval

80% match2025

Unsupervised Text Representation Learning via Instruction-Tuning for Zero-Shot Dense Retrieval

80% match2024

QAEA-DR: A Unified Text Augmentation Framework for Dense Retrieval

79% match2024

Efficient fine-tuning methodology of text embedding models for information retrieval: contrastive learning penalty (clp)

79% match1 repo112024

Cite this paper

@article{bai2024multitask,
  title  = {M3: A Multi-Task Mixed-Objective Learning Framework for Open-Domain Multi-Hop Dense Sentence Retrieval},
  author = {Yang Bai and Anthony Colas and Christan Grant and Daisy Zhe Wang},
  year   = {2024},
  eprint = {2403.14074},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2403.14074v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2403.14074.svg)](https://codeofpaper.com/paper/2403.14074)

Discussion