A Comparative Study of Self-supervised Speech Representation Based Voice Conversion

Wen-Chin Huang, Shu-Wen Yang, Tomoki Hayashi, Tomoki Toda
7/10/2022
cs.SDcs.LGeess.AS

No confident code implementation found for this paper. 1 possible match can be shown below.

Abstract

We present a large-scale comparative study of self-supervised speech representation (S3R)-based voice conversion (VC). In the context of recognition-synthesis VC, S3Rs are attractive owing to their potential to replace expensive supervised representations such as phonetic posteriorgrams (PPGs), which are commonly adopted by state-of-the-art VC systems. Using S3PRL-VC, an open-source VC software we previously developed, we provide a series of in-depth objective and subjective analyses under three VC settings: intra-/cross-lingual any-to-one (A2O) and any-to-any (A2A) VC, using the voice conversion challenge 2020 (VCC2020) dataset. We investigated S3R-based VC in various aspects, including model type, multilinguality, and supervision. We also studied the effect of a post-discretization process with k-means clustering and showed how it improves in the A2A setting. Finally, the comparison with state-of-the-art VC systems demonstrates the competitiveness of S3R-based VC and also sheds light on the possible improving directions.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. 1 weaker match is hidden by default — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Adversarial Speaker Disentanglement Using Unannotated External Data for Self-supervised Representation Based Voice Conversion

89% match2023

Non-Contrastive Self-supervised Learning for Utterance-Level Information Extraction from Speech

85% match2022

A Comparative Study of Voice Conversion Models with Large-Scale Speech and Singing Data: The T13 Systems for the Singing Voice Conversion Challenge 2023

85% match2023

Iteratively Improving Speech Recognition and Voice Conversion

84% match2023

Self-supervised Adaptive Pre-training of Multilingual Speech Models for Language and Dialect Identification

84% match2023

DinoSR: Self-Distillation and Online Clustering for Self-supervised Speech Representation Learning

83% match1 repo532026

Cite this paper

@article{huang2022comparative,
  title  = {A Comparative Study of Self-supervised Speech Representation Based Voice Conversion},
  author = {Wen-Chin Huang and Shu-Wen Yang and Tomoki Hayashi and Tomoki Toda},
  year   = {2022},
  eprint = {2207.04356},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2207.04356v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2207.04356.svg)](https://codeofpaper.com/paper/2207.04356)

Discussion