Multilingual E5 Text Embeddings: A Technical Report

Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder +1 more
2/8/2024
cs.CLcs.IR

Official code: microsoft/unilm (22,168 stars, MIT, last commit 2026-01-23).

Abstract

This technical report presents the training methodology and evaluation results of the open-source multilingual E5 text embedding models, released in mid-2023. Three embedding models of different sizes (small / base / large) are provided, offering a balance between the inference efficiency and embedding quality. The training procedure adheres to the English E5 model recipe, involving contrastive pre-training on 1 billion multilingual text pairs, followed by fine-tuning on a combination of labeled datasets. Additionally, we introduce a new instruction-tuned embedding model, whose performance is on par with state-of-the-art, English-only models of similar sizes. Information regarding the model release can be found at https://github.com/microsoft/unilm/tree/master/e5 .

View on arXivView PDF

Code Implementations(1)

microsoft/unilmOfficial100%
MIT

Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities

22,1682,703C, C++Jul 23, 20199 months agoMIT
beitbeit-3bitnetdeepnetdocument-ai+15 more

Similar Papers

SkMTEB: Slovak Massive Text Embedding Benchmark and Model Adaptation

83% match1 repo22026

E5-V: Universal Embeddings with Multimodal Large Language Models

82% match1 repo2752024

MMTEB: Massive Multilingual Text Embedding Benchmark

81% match1 repo3.5k2025

The Scandinavian Embedding Benchmarks: Comprehensive Assessment of Multilingual and Monolingual Text Embedding

81% match2 repos3.5k2024

xVLM2Vec: Adapting LVLM-based embedding models to multilinguality using Self-Knowledge Distillation

81% match1 repo2025

Llama-Embed-Nemotron-8B: A Universal Text Embedding Model for Multilingual and Cross-Lingual Tasks

81% match2025

Cite this paper

@article{wang2024multilingual,
  title  = {Multilingual E5 Text Embeddings: A Technical Report},
  author = {Liang Wang and Nan Yang and Xiaolong Huang and Linjun Yang and Rangan Majumder and Furu Wei},
  year   = {2024},
  eprint = {2402.05672},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2402.05672v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2402.05672.svg)](https://codeofpaper.com/paper/2402.05672)

Discussion