Adaptation of Whisper models to child speech recognition

Rishabh Jain, Andrei Barcovschi, Mariam Yiwere, Peter Corcoran, Horia Cucu
7/24/2023
eess.AScs.AI

No official code found. Best implementation: rishabhjain16/whisper_child_asr (12 stars, last commit 2023-05-23).

Abstract

Automatic Speech Recognition (ASR) systems often struggle with transcribing child speech due to the lack of large child speech datasets required to accurately train child-friendly ASR models. However, there are huge amounts of annotated adult speech datasets which were used to create multilingual ASR models, such as Whisper. Our work aims to explore whether such models can be adapted to child speech to improve ASR for children. In addition, we compare Whisper child-adaptations with finetuned self-supervised models, such as wav2vec2. We demonstrate that finetuning Whisper on child speech yields significant improvements in ASR performance on child speech, compared to non finetuned Whisper models. Additionally, utilizing self-supervised Wav2vec2 models that have been finetuned on child speech outperforms Whisper finetuning.

View on arXivView PDF

Code Implementations(1)

rishabhjain16/whisper_child_asrCommunity100%
120Feb 22, 20233 years ago

Similar Papers

A comparative analysis between Conformer-Transducer, Whisper, and wav2vec2 for improving the child speech recognition

92% match2023

Improving child speech recognition with augmented child-like speech

86% match2024

Zero-Shot KWS for Children's Speech using Layer-Wise Features from SSL Models

86% match2025

BabySLM: language-acquisition-friendly benchmark of self-supervised spoken language models

85% match1 repo202023

Benchmarking Children's ASR with Supervised and Self-supervised Speech Foundation Models

85% match1 repo82024

BabyHuBERT: Multilingual Self-Supervised Learning for Segmenting Speakers in Child-Centered Long-Form Recordings

84% match2025

Cite this paper

@article{jain2023adaptation,
  title  = {Adaptation of Whisper models to child speech recognition},
  author = {Rishabh Jain and Andrei Barcovschi and Mariam Yiwere and Peter Corcoran and Horia Cucu},
  year   = {2023},
  eprint = {2307.13008},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2307.13008v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2307.13008.svg)](https://codeofpaper.com/paper/2307.13008)

Discussion