Comparison of Multilingual Self-Supervised and Weakly-Supervised Speech Pre-Training for Adaptation to Unseen Languages

Andrew Rouditchenko, Sameer Khurana, Samuel Thomas, Rogerio Feris, Leonid Karlinsky +4 more
5/21/2023
cs.CLcs.SDeess.AS

No confident code implementation found for this paper.

Abstract

Recent models such as XLS-R and Whisper have made multilingual speech technologies more accessible by pre-training on audio from around 100 spoken languages each. However, there are thousands of spoken languages worldwide, and adapting to new languages is an important problem. In this work, we aim to understand which model adapts better to languages unseen during pre-training. We fine-tune both models on 13 unseen languages and 18 seen languages. Our results show that the number of hours seen per language and language family during pre-training is predictive of how the models compare, despite the significant differences in the pre-training methods.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. Any repos shown below are weak matches — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Sabiá: Portuguese Large Language Models

87% match2023

When Is Multilinguality a Curse? Language Modeling for 250 High- and Low-Resource Languages

87% match1 repo102023

Language Exposure Matters: Distribution and Order in Multilingual BabyLM Pre-training

86% match2026

Learning from Child-Directed Speech in Two-Language Scenarios: A French-English Case Study

85% match2026

Closing the Quality Gap in Low-Resource Text-to-Speech: LoRA Fine-Tuning of VoxCPM2 for Khmer and Korean

85% match2026

Polyglot-Lion: Efficient Multilingual ASR for Singapore via Balanced Fine-Tuning of Qwen3-ASR

85% match1 repo72026

Cite this paper

@article{rouditchenko2023comparison,
  title  = {Comparison of Multilingual Self-Supervised and Weakly-Supervised Speech Pre-Training for Adaptation to Unseen Languages},
  author = {Andrew Rouditchenko and Sameer Khurana and Samuel Thomas and Rogerio Feris and Leonid Karlinsky and Hilde Kuehne and David Harwath and Brian Kingsbury and James Glass},
  year   = {2023},
  eprint = {2305.12606},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2305.12606v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2305.12606.svg)](https://codeofpaper.com/paper/2305.12606)

Discussion