Phoneme-Level BERT for Enhanced Prosody of Text-to-Speech with Grapheme Predictions

Yinghao Aaron Li, Cong Han, Xilin Jiang, Nima Mesgarani
1/20/2023
cs.CLcs.SDeess.AS

No official code found. Best implementation: yl4579/PL-BERT (269 stars, MIT, last commit 2025-01-13). 1 more below.

Abstract

Large-scale pre-trained language models have been shown to be helpful in improving the naturalness of text-to-speech (TTS) models by enabling them to produce more naturalistic prosodic patterns. However, these models are usually word-level or sup-phoneme-level and jointly trained with phonemes, making them inefficient for the downstream TTS task where only phonemes are needed. In this work, we propose a phoneme-level BERT (PL-BERT) with a pretext task of predicting the corresponding graphemes along with the regular masked phoneme predictions. Subjective evaluations show that our phoneme-level BERT encoder has significantly improved the mean opinion scores (MOS) of rated naturalness of synthesized speech compared with the state-of-the-art (SOTA) StyleTTS baseline on out-of-distribution (OOD) texts.

View on arXivView PDF

Code Implementations(2)

yl4579/PL-BERTCommunity82%
MIT

Phoneme-Level BERT for Enhanced Prosody of Text-to-Speech with Grapheme Predictions

26955Python, Jupyter NotebookJan 24, 20231 years agoMIT
bertbert-modeltext-to-speechtts
Fatlonder/PL-BERTCommunity75%
MIT

Phoneme-Level BERT for Enhanced Prosody of Text-to-Speech with Grapheme Predictions

00Python, Jupyter NotebookNov 2, 20241 years agoMIT

Similar Papers

XPhoneBERT: A Pre-trained Multilingual Model for Phoneme Representations for Text-to-Speech

86% match1 repo3582023

ASR-Generated Text for Language Model Pre-training Applied to Speech Tasks

84% match2022

SoundChoice: Grapheme-to-Phoneme Models with Semantic Disambiguation

84% match1 repo11.8k2022

Closing the Quality Gap in Low-Resource Text-to-Speech: LoRA Fine-Tuning of VoxCPM2 for Khmer and Korean

83% match2026

A Comparative Analysis of Pretrained Language Models for Text-to-Speech

83% match2023

Data-driven grapheme-to-phoneme representations for a lexicon-free text-to-speech

82% match2024

Cite this paper

@article{li2023phonemelevel,
  title  = {Phoneme-Level BERT for Enhanced Prosody of Text-to-Speech with Grapheme Predictions},
  author = {Yinghao Aaron Li and Cong Han and Xilin Jiang and Nima Mesgarani},
  year   = {2023},
  eprint = {2301.08810},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2301.08810v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2301.08810.svg)](https://codeofpaper.com/paper/2301.08810)

Discussion