Learn and Don't Forget: Adding a New Language to ASR Foundation Models

Mengjie Qian, Siyuan Tang, Rao Ma, Kate M. Knill, Mark J. F. Gales
7/9/2024
eess.AScs.CLcs.LG

No confident code implementation found for this paper.

Abstract

Foundation ASR models often support many languages, e.g. 100 languages in Whisper. However, there has been limited work on integrating an additional, typically low-resource, language, while maintaining performance on the original language set. Fine-tuning, while simple, may degrade the accuracy of the original set. We compare three approaches that exploit adaptation parameters: soft language code tuning, train only the language code; soft prompt tuning, train prepended tokens; and LoRA where a small set of additional parameters are optimised. Elastic Weight Consolidation (EWC) offers an alternative compromise with the potential to maintain performance in specific target languages. Results show that direct fine-tuning yields the best performance for the new language but degrades existing language capabilities. EWC can address this issue for specific languages. If only adaptation parameters are used, the language capabilities are maintained but at the cost of performance in the new language.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. Any repos shown below are weak matches — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

A Parameter-efficient Language Extension Framework for Multilingual ASR

84% match2024

Efficient Adapter Finetuning for Tail Languages in Streaming Multilingual ASR

83% match2024

Efficient Adaptation of Multilingual Models for Japanese ASR

83% match1 repo92024

Towards Rehearsal-Free Multilingual ASR: A LoRA-based Case Study on Whisper

83% match2024

Language-Aware Prompt Tuning for Parameter-Efficient Seamless Language Expansion in Multilingual ASR

83% match2025

Adapting Whisper for Parameter-efficient Code-Switching Speech Recognition via Soft Prompt Tuning

83% match2025

Cite this paper

@article{qian2024learn,
  title  = {Learn and Don't Forget: Adding a New Language to ASR Foundation Models},
  author = {Mengjie Qian and Siyuan Tang and Rao Ma and Kate M. Knill and Mark J. F. Gales},
  year   = {2024},
  eprint = {2407.06800},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2407.06800v3}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2407.06800.svg)](https://codeofpaper.com/paper/2407.06800)

Discussion