TTSDS -- Text-to-Speech Distribution Score

Christoph Minixhofer, Ondřej Klejch, Peter Bell
7/17/2024
eess.AScs.CLcs.LG

No official code found. Best implementation: ttsds/ttsds (101 stars, last commit 2026-02-20).

Abstract

Many recently published Text-to-Speech (TTS) systems produce audio close to real speech. However, TTS evaluation needs to be revisited to make sense of the results obtained with the new architectures, approaches and datasets. We propose evaluating the quality of synthetic speech as a combination of multiple factors such as prosody, speaker identity, and intelligibility. Our approach assesses how well synthetic speech mirrors real speech by obtaining correlates of each factor and measuring their distance from both real speech datasets and noise datasets. We benchmark 35 TTS systems developed between 2008 and 2024 and show that our score computed as an unweighted average of factors strongly correlates with the human evaluations from each time period.

View on arXivView PDF

Code Implementations(1)

ttsds/ttsdsCommunity100%

The TTSDS benchmark evaluates synthetic speech quality by considering prosody, speaker identity, and intelligibility, comparing these factors with real speech and noise datasets.

1017PythonJul 18, 20248 months ago

Similar Papers

TTSDS2: Resources and Benchmark for Evaluating Human-Quality Text to Speech Systems

92% match2025

Evaluating Speech Synthesis by Training Recognizers on Synthetic Speech

82% match2023

On the Problem of Text-To-Speech Model Selection for Synthetic Data Generation in Automatic Speech Recognition

81% match2024

BASE TTS: Lessons from building a billion-parameter Text-to-Speech model on 100K hours of data

81% match2024

The State Of TTS: A Case Study with Human Fooling Rates

81% match2025

Audio Turing Test: Benchmarking the Human-likeness of Large Language Model-based Text-to-Speech Systems in Chinese

80% match2025

Cite this paper

@article{minixhofer2024ttsds,
  title  = {TTSDS -- Text-to-Speech Distribution Score},
  author = {Christoph Minixhofer and Ondřej Klejch and Peter Bell},
  year   = {2024},
  eprint = {2407.12707},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2407.12707v3}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2407.12707.svg)](https://codeofpaper.com/paper/2407.12707)

Discussion