PITS: Variational Pitch Inference without Fundamental Frequency for End-to-End Pitch-controllable TTS

Junhyeok Lee, Wonbin Jung, Hyunjae Cho, Jaeyeon Kim, Jaehwan Kim
2/24/2023
eess.AScs.LGcs.SD

Official code: anonymous-pits/pits (280 stars, MIT, last commit 2023-07-16).

Abstract

Previous pitch-controllable text-to-speech (TTS) models rely on directly modeling fundamental frequency, leading to low variance in synthesized speech. To address this issue, we propose PITS, an end-to-end pitch-controllable TTS model that utilizes variational inference to model pitch. Based on VITS, PITS incorporates the Yingram encoder, the Yingram decoder, and adversarial training of pitch-shifted synthesis to achieve pitch-controllability. Experiments demonstrate that PITS generates high-quality speech that is indistinguishable from ground truth speech and has high pitch-controllability without quality degradation. Code, audio samples, and demo are available at https://github.com/anonymous-pits/pits.

View on arXivView PDF

Code Implementations(1)

anonymous-pits/pitsOfficial100%
MIT

PITS: Variational Pitch Inference for End-to-end Pitch-controllable TTS without External Pitch Predictor

28033Cython, PythonFeb 20, 20233 years agoMIT

Similar Papers

Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech

81% match1 repo7.9k2021

FastPitch: Parallel Text-to-speech with Pitch Prediction

81% match2 repos14.8k2020

Audio Generation with Multiple Conditional Diffusion Model

79% match2023

PRESENT: Zero-Shot Text-to-Prosody Control

78% match1 repo142024

A Vector Quantized Approach for Text to Speech Synthesis on Real-World Spontaneous Speech

78% match1 repo2602023

Investigating Stochastic Methods for Prosody Modeling in Speech Synthesis

78% match2025

Cite this paper

@article{lee2023pits,
  title  = {PITS: Variational Pitch Inference without Fundamental Frequency for End-to-End Pitch-controllable TTS},
  author = {Junhyeok Lee and Wonbin Jung and Hyunjae Cho and Jaeyeon Kim and Jaehwan Kim},
  year   = {2023},
  eprint = {2302.12391},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2302.12391v3}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2302.12391.svg)](https://codeofpaper.com/paper/2302.12391)

Discussion