Preset-Voice Matching for Privacy Regulated Speech-to-Speech Translation Systems

Daniel Platnick, Bishoy Abdelnour, Eamon Earl, Rahul Kumar, Zahra Rezaei +2 more
7/18/2024
cs.CLcs.CRcs.LG

No confident code implementation found for this paper.

Abstract

In recent years, there has been increased demand for speech-to-speech translation (S2ST) systems in industry settings. Although successfully commercialized, cloning-based S2ST systems expose their distributors to liabilities when misused by individuals and can infringe on personality rights when exploited by media organizations. This work proposes a regulated S2ST framework called Preset-Voice Matching (PVM). PVM removes cross-lingual voice cloning in S2ST by first matching the input voice to a similar prior consenting speaker voice in the target-language. With this separation, PVM avoids cloning the input speaker, ensuring PVM systems comply with regulations and reduce risk of misuse. Our results demonstrate PVM can significantly improve S2ST system run-time in multi-speaker settings and the naturalness of S2ST synthesized speech. To our knowledge, PVM is the first explicitly regulated S2ST framework leveraging similarly-matched preset-voices for dynamic S2ST tasks.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. Any repos shown below are weak matches — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

E2E-VGuard: Adversarial Prevention for Production LLM-based End-To-End Speech Synthesis

81% match1 repo122025

TransVIP: Speech to Speech Translation System with Voice and Isochrony Preservation

78% match1 repo1642026

GenVC: Self-Supervised Zero-Shot Voice Conversion

78% match2025

Do Not Mimic My Voice: Speaker Identity Unlearning for Zero-Shot Text-to-Speech

78% match1 repo72026

HLTCOE JHU Submission to the Voice Privacy Challenge 2024

78% match2024

SimulS2S-LLM: Unlocking Simultaneous Inference of Speech LLMs for Speech-to-Speech Translation

78% match2025

Cite this paper

@article{platnick2024presetvoice,
  title  = {Preset-Voice Matching for Privacy Regulated Speech-to-Speech Translation Systems},
  author = {Daniel Platnick and Bishoy Abdelnour and Eamon Earl and Rahul Kumar and Zahra Rezaei and Thomas Tsangaris and Faraj Lagum},
  year   = {2024},
  eprint = {2407.13153},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2407.13153v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2407.13153.svg)](https://codeofpaper.com/paper/2407.13153)

Discussion