Hear Your Face: Face-based voice conversion with F0 estimation

Jaejun Lee, Yoori Oh, Injune Hwang, Kyogu Lee
8/19/2024
cs.SDcs.CVeess.AS

No confident code implementation found for this paper. 1 possible match can be shown below.

Abstract

This paper delves into the emerging field of face-based voice conversion, leveraging the unique relationship between an individual's facial features and their vocal characteristics. We present a novel face-based voice conversion framework that particularly utilizes the average fundamental frequency of the target speaker, derived solely from their facial images. Through extensive analysis, our framework demonstrates superior speech generation quality and the ability to align facial features with voice characteristics, including tracking of the target speaker's fundamental frequency.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. 1 weaker match is hidden by default — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Seeing Your Speech Style: A Novel Zero-Shot Identity-Disentanglement Face-based Voice Conversion

85% match2024

KAN-Based Fusion of Dual-Domain for Audio-Driven Facial Landmarks Generation

83% match1 repo322024

Zero-Shot Face-to-Speech Synthesis via Latent Space Adaptation of a Style-Diffusion TTS Model

82% match2026

UniFLG: Unified Facial Landmark Generator from Text or Speech

82% match2023

Realistic Speech-to-Face Generation with Speech-Conditioned Latent Diffusion Model with Face Prior

82% match2023

Rethinking Voice-Face Correlation: A Geometry View

82% match1 repo82023

Cite this paper

@article{lee2024hear,
  title  = {Hear Your Face: Face-based voice conversion with F0 estimation},
  author = {Jaejun Lee and Yoori Oh and Injune Hwang and Kyogu Lee},
  year   = {2024},
  eprint = {2408.09802},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2408.09802v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2408.09802.svg)](https://codeofpaper.com/paper/2408.09802)

Discussion