Investigating the Effect of Parallel Data in the Cross-Lingual Transfer for Vision-Language Encoders

Andrei-Alexandru Manea, Jindřich Libovický
4/30/2025
cs.CL

No confident code implementation found for this paper.

Abstract

Most pre-trained Vision-Language (VL) models and training data for the downstream tasks are only available in English. Therefore, multilingual VL tasks are solved using cross-lingual transfer: fine-tune a multilingual pre-trained model or transfer the text encoder using parallel data. We study the alternative approach: transferring an already trained encoder using parallel data. We investigate the effect of parallel data: domain and the number of languages, which were out of focus in previous work. Our results show that even machine-translated task data are the best on average, caption-like authentic parallel data outperformed it in some languages. Further, we show that most languages benefit from multilingual training.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. Any repos shown below are weak matches — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Just Go Parallel: Improving the Multilingual Capabilities of Large Language Models

86% match2025

Language-Specific Layer Matters: Efficient Multilingual Enhancement for Large Vision-Language Models

86% match2025

Centurio: On Drivers of Multilingual Ability of Large Vision-Language Model

86% match1 repo62025

Multilingual VLM Training: Adapting an English-Trained VLM to French

86% match2025

Rethinking Multilingual Vision-Language Translation: Dataset, Evaluation, and Adaptation

84% match2025

TowerVision: Understanding and Improving Multilinguality in Vision-Language Models

84% match2025

Cite this paper

@article{manea2025investigating,
  title  = {Investigating the Effect of Parallel Data in the Cross-Lingual Transfer for Vision-Language Encoders},
  author = {Andrei-Alexandru Manea and Jindřich Libovický},
  year   = {2025},
  eprint = {2504.21681},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2504.21681v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2504.21681.svg)](https://codeofpaper.com/paper/2504.21681)

Discussion