Towards Real-World Streaming Speech Translation for Code-Switched Speech

Belen Alastruey, Matthias Sperber, Christian Gollan, Dominic Telaar, Tim Ng +1 more
10/19/2023
cs.CL

No official code found. Best implementation: apple/ml-codeswitching-translations (9 stars, last commit 2023-10-11).

Abstract

Code-switching (CS), i.e. mixing different languages in a single sentence, is a common phenomenon in communication and can be challenging in many Natural Language Processing (NLP) settings. Previous studies on CS speech have shown promising results for end-to-end speech translation (ST), but have been limited to offline scenarios and to translation to one of the languages present in the source (\textit{monolingual transcription}). In this paper, we focus on two essential yet unexplored areas for real-world CS speech translation: streaming settings, and translation to a third language (i.e., a language not included in the source). To this end, we extend the Fisher and Miami test and validation datasets to include new targets in Spanish and German. Using this data, we train a model for both offline and streaming ST and we establish baseline results for the two settings mentioned earlier.

View on arXivView PDF

Code Implementations(1)

apple/ml-codeswitching-translationsCommunity100%
91Aug 28, 20233 years agoNOASSERTION

Similar Papers

Adapting Offline Speech Translation Models for Streaming with Future-Aware Distillation and Inference

85% match1 repo62023

Code-Switching Text Generation and Injection in Mandarin-English ASR

84% match2023

Code-Switching Detection via Language-Selective Dimensions in Speech Foundation Models

84% match2026

Seamless: Multilingual Expressive and Streaming Speech Translation

83% match1 repo11.9k2023

Cross-Lingual Transfer Learning for Speech Translation

82% match2024

The Effect of Alignment Objectives on Code-Switching Translation

82% match2023

Cite this paper

@article{alastruey2023towards,
  title  = {Towards Real-World Streaming Speech Translation for Code-Switched Speech},
  author = {Belen Alastruey and Matthias Sperber and Christian Gollan and Dominic Telaar and Tim Ng and Aashish Agarwal},
  year   = {2023},
  eprint = {2310.12648},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2310.12648v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2310.12648.svg)](https://codeofpaper.com/paper/2310.12648)

Discussion