Distillation for Multilingual Information Retrieval

Eugene Yang, Dawn Lawrie, James Mayfield
5/2/2024
cs.IRcs.CL

No official code found. Best implementation: hltcoe/ColBERT-X (73 stars, MIT, last commit 2025-06-23).

Abstract

Recent work in cross-language information retrieval (CLIR), where queries and documents are in different languages, has shown the benefit of the Translate-Distill framework that trains a cross-language neural dual-encoder model using translation and distillation. However, Translate-Distill only supports a single document language. Multilingual information retrieval (MLIR), which ranks a multilingual document collection, is harder to train than CLIR because the model must assign comparable relevance scores to documents in different languages. This work extends Translate-Distill and propose Multilingual Translate-Distill (MTD) for MLIR. We show that ColBERT-X models trained with MTD outperform their counterparts trained ith Multilingual Translate-Train, which is the previous state-of-the-art training approach, by 5% to 25% in nDCG@20 and 15% to 45% in MAP. We also show that the model is robust to the way languages are mixed in training batches. Our implementation is available on GitHub.

View on arXivView PDF

Code Implementations(1)

hltcoe/ColBERT-XCommunity100%
MIT

CLIR version of ColBERT

7314C++, CudaJan 20, 20221 years agoMIT

Similar Papers

Translate-Distill: Learning Cross-Language Dense Retrieval by Translation and Distillation

92% match1 repo732024

What Drives Cross-lingual Ranking? Retrieval Approaches with Multilingual Language Models

86% match2025

Multi-Hypothesis Distillation of Multilingual Neural Translation Models for Low-Resource Languages

83% match2025

ColBERT-XM: A Modular Multi-Vector Representation Model for Zero-Shot Multilingual Information Retrieval

83% match2024

Bridging Language Gaps: Advances in Cross-Lingual Information Retrieval with Multilingual LLMs

83% match2025

Distilling Multilingual Vision-Language Models: When Smaller Models Stay Multilingual

83% match2025

Cite this paper

@article{yang2024distillation,
  title  = {Distillation for Multilingual Information Retrieval},
  author = {Eugene Yang and Dawn Lawrie and James Mayfield},
  year   = {2024},
  eprint = {2405.00977},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2405.00977v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2405.00977.svg)](https://codeofpaper.com/paper/2405.00977)

Discussion