Modular Speech-to-Text Translation for Zero-Shot Cross-Modal Transfer

Paul-Ambroise Duquenne, Holger Schwenk, Benoît Sagot
10/5/2023
cs.CL

No confident code implementation found for this paper.

Abstract

Recent research has shown that independently trained encoders and decoders, combined through a shared fixed-size representation, can achieve competitive performance in speech-to-text translation. In this work, we show that this type of approach can be further improved with multilingual training. We observe significant improvements in zero-shot cross-modal speech translation, even outperforming a supervised approach based on XLSR for several languages.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. Any repos shown below are weak matches — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Textless Unit-to-Unit training for Many-to-Many Multilingual Speech-to-Speech Translation

89% match1 repo312023

Cross-Modal Multi-Tasking for Speech-to-Text Translation via Hard Parameter Sharing

87% match2023

Improved Cross-Lingual Transfer Learning For Automatic Speech Translation

87% match2023

Soft Alignment of Modality Space for End-to-end Speech Translation

87% match2023

ComSL: A Composite Speech-Language Model for End-to-End Speech-to-Text Translation

87% match1 repo112026

Multilingual Speech-to-Speech Translation into Multiple Target Languages

87% match2023

Cite this paper

@article{duquenne2023modular,
  title  = {Modular Speech-to-Text Translation for Zero-Shot Cross-Modal Transfer},
  author = {Paul-Ambroise Duquenne and Holger Schwenk and Benoît Sagot},
  year   = {2023},
  eprint = {2310.03724},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2310.03724v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2310.03724.svg)](https://codeofpaper.com/paper/2310.03724)

Discussion