Zero-Shot Multi-Speaker Text-To-Speech with State-of-the-art Neural Speaker Embeddings

Erica Cooper, Cheng-I Lai, Yusuke Yasuda, Fuming Fang, Xin Wang +2 more
10/23/2019
eess.AS

Official code: nii-yamagishilab/multi-speaker-tacotron (266 stars, BSD-3-Clause, last commit 2022-03-29).

Abstract

While speaker adaptation for end-to-end speech synthesis using speaker embeddings can produce good speaker similarity for speakers seen during training, there remains a gap for zero-shot adaptation to unseen speakers. We investigate multi-speaker modeling for end-to-end text-to-speech synthesis and study the effects of different types of state-of-the-art neural speaker embeddings on speaker similarity for unseen speakers. Learnable dictionary encoding-based speaker embeddings with angular softmax loss can improve equal error rates over x-vectors in a speaker verification task; these embeddings also improve speaker similarity and naturalness for unseen speakers when used for zero-shot adaptation to new speakers in end-to-end speech synthesis.

View on arXivView PDF

Code Implementations(1)

nii-yamagishilab/multi-speaker-tacotronOfficial100%
BSD-3-Clause

VCTK multi-speaker tacotron for ICASSP 2020

26642Shell, PythonOct 20, 20194 years agoBSD-3-Clause

Similar Papers

Generalizable Zero-Shot Speaker Adaptive Speech Synthesis with Disentangled Representations

86% match2023

Analyzing And Improving Neural Speaker Embeddings for ASR

85% match2023

Automatic Evaluation of Speaker Similarity

85% match2022

Problem-Agnostic Speech Embeddings for Multi-Speaker Text-to-Speech with SampleRNN

84% match2019

SC-GlowTTS: an Efficient Zero-Shot Multi-Speaker Text-To-Speech Model

83% match2 repos46.1k2021

LoRP-TTS: Low-Rank Personalized Text-To-Speech

83% match2025

Cite this paper

@article{cooper2019zeroshot,
  title  = {Zero-Shot Multi-Speaker Text-To-Speech with State-of-the-art Neural Speaker Embeddings},
  author = {Erica Cooper and Cheng-I Lai and Yusuke Yasuda and Fuming Fang and Xin Wang and Nanxin Chen and Junichi Yamagishi},
  year   = {2019},
  eprint = {1910.10838},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/1910.10838}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/1910.10838.svg)](https://codeofpaper.com/paper/1910.10838)

Discussion