XTTS: a Massively Multilingual Zero-Shot Text-to-Speech Model

Edresson Casanova, Kelly Davis, Eren Gölge, Görkem Göknar, Iulian Gulea +6 more
6/7/2024
eess.AScs.CLcs.SD

No official code found. Best implementation: Edresson/ZS-TTS-Evaluation (45 stars, MIT, last commit 2024-09-19).

Abstract

Most Zero-shot Multi-speaker TTS (ZS-TTS) systems support only a single language. Although models like YourTTS, VALL-E X, Mega-TTS 2, and Voicebox explored Multilingual ZS-TTS they are limited to just a few high/medium resource languages, limiting the applications of these models in most of the low/medium resource languages. In this paper, we aim to alleviate this issue by proposing and making publicly available the XTTS system. Our method builds upon the Tortoise model and adds several novel modifications to enable multilingual training, improve voice cloning, and enable faster training and inference. XTTS was trained in 16 languages and achieved state-of-the-art (SOTA) results in most of them.

View on arXivView PDF

Code Implementations(1)

Edresson/ZS-TTS-EvaluationCommunity100%
MIT
452PythonJan 28, 20242 years agoMIT

Similar Papers

Meta Learning Text-to-Speech Synthesis in over 7000 Languages

84% match1 repo2.2k2024

OmniVoice: Towards Omnilingual Zero-Shot Text-to-Speech with Diffusion Language Models

84% match2026

MParrotTTS: Multilingual Multi-speaker Text to Speech Synthesis in Low Resource Setting

83% match2023

Generalized Multilingual Text-to-Speech Generation with Language-Aware Style Adaptation

82% match2025

Scaling NVIDIA's Multi-speaker Multi-lingual TTS Systems with Zero-Shot TTS to Indic Languages

82% match2024

RosettaSpeech: Zero-Shot Speech-to-Speech Translation without Parallel Speech

81% match2025

Cite this paper

@article{casanova2024xtts,
  title  = {XTTS: a Massively Multilingual Zero-Shot Text-to-Speech Model},
  author = {Edresson Casanova and Kelly Davis and Eren Gölge and Görkem Göknar and Iulian Gulea and Logan Hart and Aya Aljafari and Joshua Meyer and Reuben Morais and Samuel Olayemi and Julian Weber},
  year   = {2024},
  eprint = {2406.04904},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2406.04904v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2406.04904.svg)](https://codeofpaper.com/paper/2406.04904)

Discussion