FLiP: Towards understanding and interpreting multimodal multilingual sentence embeddings

Santosh Kesiraju, Bolaji Yusuf, Šimon Sedláček, Oldřich Plchot, Petr Schwarz
4/20/2026
cs.CLcs.SD

Official code: BUTSpeechFIT/FLiP (1 star, last commit 2026-04-22).

Abstract

This paper presents factorized linear projection (FLiP) models for understanding pretrained sentence embedding spaces. We train FLiP models to recover the lexical content from multilingual (LaBSE), multimodal (SONAR) and API-based (Gemini) sentence embedding spaces in several high- and mid-resource languages. We show that FLiP can recall more than 75% of lexical content from the embeddings, significantly outperforming existing non-factorized baselines. Using this as a diagnostic tool, we uncover the modality and language biases across the selected sentence encoders and provide practitioners with intrinsic insights about the encoders without relying on conventional downstream evaluation tasks. Our implementation is public https://github.com/BUTSpeechFIT/FLiP.

View on arXivView PDF

Code Implementations(1)

BUTSpeechFIT/FLiPOfficial100%

Factorized Linear Projection for Interpreting Multilingual and Multimodal Sentence Embeddings

10PythonApr 13, 20266 months agoNOASSERTION

Similar Papers

Forewarned is Forearmed: When Non-Sequential Embedding Turns Into an Anomaly Detector

86% match2026

Omnilingual SONAR: Cross-Lingual and Cross-Modal Sentence Embeddings Bridging Massively Multilingual Text and Speech

85% match2026

SONAR: Sentence-Level Multimodal and Language-Agnostic Representations

85% match3 repos9132023

From Word Vectors to Multimodal Embeddings: Techniques, Applications, and Future Directions For Large Language Models

84% match2024

Japanese SimCSE Technical Report

83% match1 repo692023

Multilingual Sentence-T5: Scalable Sentence Encoders for Multilingual Applications

83% match2024

Cite this paper

@article{kesiraju2026flip,
  title  = {FLiP: Towards understanding and interpreting multimodal multilingual sentence embeddings},
  author = {Santosh Kesiraju and Bolaji Yusuf and Šimon Sedláček and Oldřich Plchot and Petr Schwarz},
  year   = {2026},
  eprint = {2604.18109},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2604.18109v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2604.18109.svg)](https://codeofpaper.com/paper/2604.18109)

Discussion