Generative Adversarial Training for Text-to-Speech Synthesis Based on Raw Phonetic Input and Explicit Prosody Modelling

Tiberiu Boros, Stefan Daniel Dumitrescu, Ionut Mironica, Radu Chivereanu
10/14/2023
cs.LG

Official code: tiberiu44/TTS-Cube (223 stars, Apache-2.0, last commit 2024-02-24).

Abstract

We describe an end-to-end speech synthesis system that uses generative adversarial training. We train our Vocoder for raw phoneme-to-audio conversion, using explicit phonetic, pitch and duration modeling. We experiment with several pre-trained models for contextualized and decontextualized word embeddings and we introduce a new method for highly expressive character voice matching, based on discreet style tokens.

View on arXivView PDF

Code Implementations(1)

tiberiu44/TTS-CubeOfficial
Apache-2.0

End-2-end speech synthesis with recurrent neural networks

22343PythonJul 21, 20182 years agoApache-2.0
autoregressivecharacterend-2-endlong-short-term-memorylstm+7 more

Similar Papers

Multi-modal Adversarial Training for Zero-Shot Voice Cloning

86% match2024

Generative Pre-training for Speech with Flow Matching

83% match2026

A Vector Quantized Approach for Text to Speech Synthesis on Real-World Spontaneous Speech

81% match1 repo2602023

MixedG2P-T5: G2P-free Speech Synthesis for Mixed-script texts using Speech Self-Supervised Learning and Language Model

81% match2025

StyleTTS 2: Towards Human-Level Text-to-Speech through Style Diffusion and Adversarial Training with Large Speech Language Models

81% match3 repos6.4k2023

StyleSpeech: Parameter-efficient Fine Tuning for Pre-trained Controllable Text-to-Speech

81% match1 repo32024

Cite this paper

@article{boros2023generative,
  title  = {Generative Adversarial Training for Text-to-Speech Synthesis Based on Raw Phonetic Input and Explicit Prosody Modelling},
  author = {Tiberiu Boros and Stefan Daniel Dumitrescu and Ionut Mironica and Radu Chivereanu},
  year   = {2023},
  eprint = {2310.09636},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2310.09636v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2310.09636.svg)](https://codeofpaper.com/paper/2310.09636)

Discussion