Flowtron: an Autoregressive Flow-based Generative Network for Text-to-Speech Synthesis

Rafael Valle, Kevin Shih, Ryan Prenger, Bryan Catanzaro
5/12/2020
cs.SDcs.CLcs.LG

Official code: NVIDIA/flowtron (895 stars, Apache-2.0, last commit 2023-07-06).

Abstract

In this paper we propose Flowtron: an autoregressive flow-based generative network for text-to-speech synthesis with control over speech variation and style transfer. Flowtron borrows insights from IAF and revamps Tacotron in order to provide high-quality and expressive mel-spectrogram synthesis. Flowtron is optimized by maximizing the likelihood of the training data, which makes training simple and stable. Flowtron learns an invertible mapping of data to a latent space that can be manipulated to control many aspects of speech synthesis (pitch, tone, speech rate, cadence, accent). Our mean opinion scores (MOS) show that Flowtron matches state-of-the-art TTS models in terms of speech quality. In addition, we provide results on control of speech variation, interpolation between samples and style transfer between speakers seen and unseen during training. Code and pre-trained models will be made publicly available at https://github.com/NVIDIA/flowtron

View on arXivView PDF

Code Implementations(1)

NVIDIA/flowtronOfficial100%
Apache-2.0

Flowtron is an auto-regressive flow-based generative network for text to speech synthesis with control over speech variation and style transfer

895174Python, DockerfileApr 30, 20203 years agoApache-2.0
speech-synthesis

Similar Papers

FloWaveNet : A Generative Flow for Raw Audio

87% match1 repo4902018

HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis

87% match1 repo2.4k2020

PortaSpeech: Portable and High-Quality Generative Text-to-Speech

87% match1 repo1.0k2021

Glow-TTS: A Generative Flow for Text-to-Speech via Monotonic Alignment Search

86% match1 repo7132020

DEX-TTS: Diffusion-based EXpressive Text-to-Speech with Style Modeling on Time Variability

85% match1 repo1082024

Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech

84% match1 repo7.9k2021

Cite this paper

@article{valle2020flowtron,
  title  = {Flowtron: an Autoregressive Flow-based Generative Network for Text-to-Speech Synthesis},
  author = {Rafael Valle and Kevin Shih and Ryan Prenger and Bryan Catanzaro},
  year   = {2020},
  eprint = {2005.05957},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/2005.05957}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2005.05957.svg)](https://codeofpaper.com/paper/2005.05957)

Discussion