On the Relevance of Phoneme Duration Variability of Synthesized Training Data for Automatic Speech Recognition

Nick Rossenbach, Benedikt Hilmes, Ralf Schlüter
10/12/2023
cs.CLcs.SDeess.AS

No confident code implementation found for this paper.

Abstract

Synthetic data generated by text-to-speech (TTS) systems can be used to improve automatic speech recognition (ASR) systems in low-resource or domain mismatch tasks. It has been shown that TTS-generated outputs still do not have the same qualities as real data. In this work we focus on the temporal structure of synthetic data and its relation to ASR training. By using a novel oracle setup we show how much the degradation of synthetic data quality is influenced by duration modeling in non-autoregressive (NAR) TTS. To get reference phoneme durations we use two common alignment methods, a hidden Markov Gaussian-mixture model (HMM-GMM) aligner and a neural connectionist temporal classification (CTC) aligner. Using a simple algorithm based on random walks we shift phoneme duration distributions of the TTS system closer to real durations, resulting in an improvement of an ASR system using synthetic data in a semi-supervised setting.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. Any repos shown below are weak matches — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Adaptive Duration Model for Text Speech Alignment

85% match2025

Timestamped Embedding-Matching Acoustic-to-Word CTC ASR

83% match2023

On the Effect of Purely Synthetic Training Data for Different Automatic Speech Recognition Architectures

83% match2024

MRI2Speech: Speech Synthesis from Articulatory Movements Recorded by Real-time MRI

83% match2024

High-Fidelity Speech Synthesis with Minimal Supervision: All Using Diffusion Models

82% match2023

A CTC Alignment-based Non-autoregressive Transformer for End-to-end Automatic Speech Recognition

82% match2023

Cite this paper

@article{rossenbach2023relevance,
  title  = {On the Relevance of Phoneme Duration Variability of Synthesized Training Data for Automatic Speech Recognition},
  author = {Nick Rossenbach and Benedikt Hilmes and Ralf Schlüter},
  year   = {2023},
  eprint = {2310.08132},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2310.08132v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2310.08132.svg)](https://codeofpaper.com/paper/2310.08132)

Discussion