FastPitch: Parallel Text-to-speech with Pitch Prediction

Adrian Łańcucki
6/11/2020
eess.AScs.CLcs.LG

Official code: NVIDIA/DeepLearningExamples (14,834 stars, PyTorch, last commit 2024-08-12). 1 more implementation below.

Abstract

We present FastPitch, a fully-parallel text-to-speech model based on FastSpeech, conditioned on fundamental frequency contours. The model predicts pitch contours during inference. By altering these predictions, the generated speech can be more expressive, better match the semantic of the utterance, and in the end more engaging to the listener. Uniformly increasing or decreasing pitch with FastPitch generates speech that resembles the voluntary modulation of voice. Conditioning on frequency contours improves the overall quality of synthesized speech, making it comparable to state-of-the-art. It does not introduce an overhead, and FastPitch retains the favorable, fully-parallel Transformer architecture, with over 900x real-time factor for mel-spectrogram synthesis of a typical utterance.

View on arXivView PDF

Code Implementations(2)

NVIDIA/DeepLearningExamplesOfficial100%
PyTorch

State-of-the-Art Deep Learning scripts organized by models - easy to train and deploy with reproducible accuracy and performance on enterprise-grade infrastructure.

14,8343,407C, C++May 2, 20182 years ago
computer-visiondeep-learningdrug-discoveryforecastinglarge-language-models+10 more
axelspringer/ForwardTacotronCommunity100%
PyTorchMIT

⏩ Generating speech in a single forward pass without any attention!

577107PythonMar 3, 20207 months agoMIT
axelspringeraideep-learningforwardtacotronpythonpytorch+3 more

Similar Papers

NaturalL2S: End-to-End High-quality Multispeaker Lip-to-Speech Synthesis with Differential Digital Signal Processing

85% match2025

SALTTS: Leveraging Self-Supervised Speech Representations for improved Text-to-Speech Synthesis

84% match2023

Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech

83% match1 repo7.9k2021

High-Fidelity Speech Synthesis with Minimal Supervision: All Using Diffusion Models

83% match2023

TontaubeV1: Streaming Text-to-Speech with Hierarchical Codec Modeling and Bounded Context

83% match2026

PortaSpeech: Portable and High-Quality Generative Text-to-Speech

83% match1 repo1.0k2021

Cite this paper

@article{acucki2020fastpitch,
  title  = {FastPitch: Parallel Text-to-speech with Pitch Prediction},
  author = {Adrian Łańcucki},
  year   = {2020},
  eprint = {2006.06873},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/2006.06873}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2006.06873.svg)](https://codeofpaper.com/paper/2006.06873)

Discussion