One-shot Voice Conversion by Separating Speaker and Content Representations with Instance Normalization

Ju-chieh Chou, Cheng-chieh Yeh, Hung-yi Lee
4/10/2019
cs.LGcs.SDeess.AS

Official code: jjery2243542/adaptive_voice_conversion (483 stars, Apache-2.0, last commit 2020-10-29).

Abstract

Recently, voice conversion (VC) without parallel data has been successfully adapted to multi-target scenario in which a single model is trained to convert the input voice to many different speakers. However, such model suffers from the limitation that it can only convert the voice to the speakers in the training data, which narrows down the applicable scenario of VC. In this paper, we proposed a novel one-shot VC approach which is able to perform VC by only an example utterance from source and target speaker respectively, and the source and target speaker do not even need to be seen during training. This is achieved by disentangling speaker and content representations with instance normalization (IN). Objective and subjective evaluation shows that our model is able to generate the voice similar to target speaker. In addition to the performance measurement, we also demonstrate that this model is able to learn meaningful speaker representations without any supervision.

View on arXivView PDF

Code Implementations(1)

jjery2243542/adaptive_voice_conversionOfficial100%
Apache-2.0
48392Shell, PythonJan 15, 20195 years agoApache-2.0

Similar Papers

Multi-target Voice Conversion without Parallel Data by Adversarially Learning Disentangled Audio Representations

84% match1 repo2442018

MaskVCT: Masked Voice Codec Transformer for Zero-Shot Voice Conversion With Increased Controllability via Multiple Guidances

83% match2025

ACE-VC: Adaptive and Controllable Voice Conversion using Explicitly Disentangled Self-supervised Speech Representations

83% match2023

YourTTS: Towards Zero-Shot Multi-Speaker TTS and Zero-Shot Voice Conversion for everyone

82% match2 repos46.1k2021

Creating New Voices using Normalizing Flows

82% match2023

Noro: Noise-Robust One-shot Voice Conversion with Hidden Speaker Representation Learning

82% match2024

Cite this paper

@article{chou2019oneshot,
  title  = {One-shot Voice Conversion by Separating Speaker and Content Representations with Instance Normalization},
  author = {Ju-chieh Chou and Cheng-chieh Yeh and Hung-yi Lee},
  year   = {2019},
  eprint = {1904.05742},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/1904.05742}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/1904.05742.svg)](https://codeofpaper.com/paper/1904.05742)

Discussion