YourTTS: Towards Zero-Shot Multi-Speaker TTS and Zero-Shot Voice Conversion for everyone

Edresson Casanova, Julian Weber, Christopher Shulby, Arnaldo Candido Junior, Eren Gölge +1 more
12/4/2021
cs.SDcs.CLeess.AS

Official code: coqui-ai/TTS (46,143 stars, MPL-2.0, PyTorch, last commit 2024-08-16). 1 more implementation below.

Abstract

YourTTS brings the power of a multilingual approach to the task of zero-shot multi-speaker TTS. Our method builds upon the VITS model and adds several novel modifications for zero-shot multi-speaker and multilingual training. We achieved state-of-the-art (SOTA) results in zero-shot multi-speaker TTS and results comparable to SOTA in zero-shot voice conversion on the VCTK dataset. Additionally, our approach achieves promising results in a target language with a single-speaker dataset, opening possibilities for zero-shot multi-speaker TTS and zero-shot voice conversion systems in low-resource languages. Finally, it is possible to fine-tune the YourTTS model with less than 1 minute of speech and achieve state-of-the-art results in voice similarity and with reasonable quality. This is important to allow synthesis for speakers with a very different voice or recording characteristics from those seen during training.

View on arXivView PDF

Code Implementations(2)

coqui-ai/TTSOfficial100%
PyTorchMPL-2.0

🐸💬 - a deep learning toolkit for Text-to-Speech, battle-tested in research and production

46,1436,169HTML, ShellMay 20, 20202 years agoMPL-2.0
deep-learningglow-ttshifiganmelganmulti-speaker-tts+14 more
Edresson/YourTTSCommunity100%

YourTTS: Towards Zero-Shot Multi-Speaker TTS and Zero-Shot Voice Conversion for everyone

1,05297Python, Jupyter NotebookOct 6, 20211 years agoNOASSERTION
speech-synthesisttsvoice-conversionzero-shot-multi-speaker-ttszero-shot-voice-conversion

Similar Papers

MaskVCT: Masked Voice Codec Transformer for Zero-Shot Voice Conversion With Increased Controllability via Multiple Guidances

88% match2025

AutoCycle-VC: Towards Bottleneck-Independent Zero-Shot Cross-Lingual Voice Conversion

86% match2023

Zero-AVSR: Zero-Shot Audio-Visual Speech Recognition with LLMs by Learning Language-Agnostic Speech Representations

86% match1 repo362025

Takin: A Cohort of Superior Quality Zero-shot Speech Generation Models

84% match2024

SC-GlowTTS: an Efficient Zero-Shot Multi-Speaker Text-To-Speech Model

84% match2 repos46.1k2021

HierSpeech++: Bridging the Gap between Semantic and Acoustic Representation of Speech by Hierarchical Variational Inference for Zero-shot Speech Synthesis

84% match1 repo1.2k2023

Cite this paper

@article{casanova2021yourtts,
  title  = {YourTTS: Towards Zero-Shot Multi-Speaker TTS and Zero-Shot Voice Conversion for everyone},
  author = {Edresson Casanova and Julian Weber and Christopher Shulby and Arnaldo Candido Junior and Eren Gölge and Moacir Antonelli Ponti},
  year   = {2021},
  eprint = {2112.02418},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/2112.02418}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2112.02418.svg)](https://codeofpaper.com/paper/2112.02418)

Discussion