Unsupervised Pre-Training For Data-Efficient Text-to-Speech On Low Resource Languages

Seongyeon Park, Myungseo Song, Bohyung Kim, Tae-Hyun Oh
3/28/2023
eess.AScs.AIcs.LG

Official code: cnaigithub/SpeechDewarping (27 stars, last commit 2023-04-27).

Abstract

Neural text-to-speech (TTS) models can synthesize natural human speech when trained on large amounts of transcribed speech. However, collecting such large-scale transcribed data is expensive. This paper proposes an unsupervised pre-training method for a sequence-to-sequence TTS model by leveraging large untranscribed speech data. With our pre-training, we can remarkably reduce the amount of paired transcribed data required to train the model for the target downstream TTS task. The main idea is to pre-train the model to reconstruct de-warped mel-spectrograms from warped ones, which may allow the model to learn proper temporal assignment relation between input and output sequences. In addition, we propose a data augmentation method that further improves the data efficiency in fine-tuning. We empirically demonstrate the effectiveness of our proposed method in low-resource language scenarios, achieving outstanding performance compared to competing methods. The code and audio samples are available at: https://github.com/cnaigithub/SpeechDewarping

View on arXivView PDF

Code Implementations(1)

cnaigithub/SpeechDewarpingOfficial100%

Official implementation of "Unsupervised Pre-training for Data-Efficient Text-to-Speech on Low Resource Languages", ICASSP 2023

275HTML, PythonMar 8, 20233 years agoNOASSERTION

Similar Papers

Textless Speech-to-Speech Translation With Limited Parallel Data

85% match1 repo102023

Improving End-to-End Speech Processing by Efficient Text Data Utilization with Latent Synthesis

85% match2023

Text-to-Audio Generation using Instruction-Tuned LLM and Latent Diffusion Model

84% match1 repo1.2k2023

BASE TTS: Lessons from building a billion-parameter Text-to-Speech model on 100K hours of data

84% match2024

Minimally-Supervised Speech Synthesis with Conditional Diffusion Model and Language Model: A Comparative Study of Semantic Coding

84% match2023

Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech

84% match1 repo7.9k2021

Cite this paper

@article{park2023unsupervised,
  title  = {Unsupervised Pre-Training For Data-Efficient Text-to-Speech On Low Resource Languages},
  author = {Seongyeon Park and Myungseo Song and Bohyung Kim and Tae-Hyun Oh},
  year   = {2023},
  eprint = {2303.15669},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2303.15669v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2303.15669.svg)](https://codeofpaper.com/paper/2303.15669)

Discussion