Time and Tokens: Benchmarking End-to-End Speech Dysfluency Detection

Xuanru Zhou, Jiachen Lian, Cheol Jun Cho, Jingwen Liu, Zongli Ye +8 more
9/20/2024
eess.AScs.AIcs.SD

No official code found. Best implementation: rorizzz/TbDD (5 stars, MIT, PyTorch, last commit 2024-10-09).

Abstract

Speech dysfluency modeling is a task to detect dysfluencies in speech, such as repetition, block, insertion, replacement, and deletion. Most recent advancements treat this problem as a time-based object detection problem. In this work, we revisit this problem from a new perspective: tokenizing dysfluencies and modeling the detection problem as a token-based automatic speech recognition (ASR) problem. We propose rule-based speech and text dysfluency simulators and develop VCTK-token, and then develop a Whisper-like seq2seq architecture to build a new benchmark with decent performance. We also systematically compare our proposed token-based methods with time-based methods, and propose a unified benchmark to facilitate future research endeavors. We open-source these resources for the broader scientific community. The project page is available at https://rorizzz.github.io/

View on arXivView PDF

Code Implementations(1)

rorizzz/TbDDCommunity67%
PyTorchMIT

Time and Tokens: Benchmarking End-to-End Speech Dysfluency Detection

50Python, Jupyter NotebookOct 5, 20242 years agoMIT
asrdeep-learningdetedysfluency-detectiondyslexia+2 more

Similar Papers

Stutter-Solver: End-to-end Multi-lingual Dysfluency Detection

88% match2024

StutterCut: Uncertainty-Guided Normalised Cut for Dysfluency Segmentation

82% match2025

A Comparative Study of Controllability, Explainability, and Performance in Dysfluency Detection Models

82% match2025

SSDM: Scalable Speech Dysfluency Modeling

79% match2026

Towards Hierarchical Spoken Language Dysfluency Modeling

79% match2024

On the Difficulty of Token-Level Modeling of Dysfluency and Fluency Shaping Artifacts

79% match2025

Cite this paper

@article{zhou2024time,
  title  = {Time and Tokens: Benchmarking End-to-End Speech Dysfluency Detection},
  author = {Xuanru Zhou and Jiachen Lian and Cheol Jun Cho and Jingwen Liu and Zongli Ye and Jinming Zhang and Brittany Morin and David Baquirin and Jet Vonk and Zoe Ezzes and Zachary Miller and Maria Luisa Gorno Tempini and Gopala Anumanchipalli},
  year   = {2024},
  eprint = {2409.13582},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2409.13582v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2409.13582.svg)](https://codeofpaper.com/paper/2409.13582)

Discussion