SC-GlowTTS: an Efficient Zero-Shot Multi-Speaker Text-To-Speech Model

Edresson Casanova, Christopher Shulby, Eren Gölge, Nicolas Michael Müller, Frederico Santos de Oliveira +4 more
4/2/2021
eess.AScs.SD

Official code: coqui-ai/TTS (46,143 stars, MPL-2.0, PyTorch, last commit 2024-08-16). 1 more implementation below.

Abstract

In this paper, we propose SC-GlowTTS: an efficient zero-shot multi-speaker text-to-speech model that improves similarity for speakers unseen during training. We propose a speaker-conditional architecture that explores a flow-based decoder that works in a zero-shot scenario. As text encoders, we explore a dilated residual convolutional-based encoder, gated convolutional-based encoder, and transformer-based encoder. Additionally, we have shown that adjusting a GAN-based vocoder for the spectrograms predicted by the TTS model on the training dataset can significantly improve the similarity and speech quality for new speakers. Our model converges using only 11 speakers, reaching state-of-the-art results for similarity with new speakers, as well as high speech quality.

View on arXivView PDF

Code Implementations(2)

coqui-ai/TTSOfficial100%
PyTorchMPL-2.0

🐸💬 - a deep learning toolkit for Text-to-Speech, battle-tested in research and production

46,1436,169HTML, ShellMay 20, 20202 years agoMPL-2.0
deep-learningglow-ttshifiganmelganmulti-speaker-tts+14 more
Edresson/SC-GlowTTSCommunity100%
MIT

SC-GlowTTS: an Efficient Zero-Shot Multi-Speaker Text-To-Speech Model

10815Jan 10, 20215 years agoMIT

Similar Papers

Takin: A Cohort of Superior Quality Zero-shot Speech Generation Models

86% match2024

An Investigation of Noise Robustness for Flow-Matching-Based Zero-Shot TTS

85% match2024

MaskVCT: Masked Voice Codec Transformer for Zero-Shot Voice Conversion With Increased Controllability via Multiple Guidances

85% match2025

Zero-Shot Voice Conversion via Content-Aware Timbre Ensemble and Conditional Flow Matching

85% match2024

Mega-TTS: Zero-Shot Text-to-Speech at Scale with Intrinsic Inductive Bias

85% match2023

AutoCycle-VC: Towards Bottleneck-Independent Zero-Shot Cross-Lingual Voice Conversion

84% match2023

Cite this paper

@article{casanova2021scglowtts,
  title  = {SC-GlowTTS: an Efficient Zero-Shot Multi-Speaker Text-To-Speech Model},
  author = {Edresson Casanova and Christopher Shulby and Eren Gölge and Nicolas Michael Müller and Frederico Santos de Oliveira and Arnaldo Candido Junior and Anderson da Silva Soares and Sandra Maria Aluisio and Moacir Antonelli Ponti},
  year   = {2021},
  eprint = {2104.05557},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/2104.05557}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2104.05557.svg)](https://codeofpaper.com/paper/2104.05557)

Discussion