Achieving Timestamp Prediction While Recognizing with Non-Autoregressive End-to-End ASR Model

Xian Shi, Yanni Chen, Shiliang Zhang, Zhijie Yan
1/29/2023
cs.SDcs.CLeess.AS

No official code found. Best implementation: modelscope/FunASR (19,439 stars, MIT, PyTorch, last commit 2026-03-17).

Abstract

Conventional ASR systems use frame-level phoneme posterior to conduct force-alignment~(FA) and provide timestamps, while end-to-end ASR systems especially AED based ones are short of such ability. This paper proposes to perform timestamp prediction~(TP) while recognizing by utilizing continuous integrate-and-fire~(CIF) mechanism in non-autoregressive ASR model - Paraformer. Foucing on the fire place bias issue of CIF, we conduct post-processing strategies including fire-delay and silence insertion. Besides, we propose to use scaled-CIF to smooth the weights of CIF output, which is proved beneficial for both ASR and TP task. Accumulated averaging shift~(AAS) and diarization error rate~(DER) are adopted to measure the quality of timestamps and we compare these metrics of proposed system and conventional hybrid force-alignment system. The experiment results over manually-marked timestamps testset show that the proposed optimization methods significantly improve the accuracy of CIF timestamps, reducing 66.7\% and 82.1\% of AAS and DER respectively. Comparing to Kaldi force-alignment trained with the same data, optimized CIF timestamps achieved 12.3\% relative AAS reduction.

View on arXivView PDF

Code Implementations(1)

modelscope/FunASRCommunity100%
PyTorchMIT

A Fundamental End-to-End Speech Recognition Toolkit and Open Source SOTA Pretrained Models, Supporting Speech Recognition, Voice Activity Detection, Text Post-processing etc.

19,4391,955C, C#Nov 24, 20227 months agoMIT
audio-visual-speech-recognitionconformerdfsmnparaformerpretrained-model+10 more

Similar Papers

In-Sync: Adaptation of Speech Aware Large Language Models for ASR with Word Level Timestamp Predictions

84% match2026

REDDIT: Correcting Model-Generated Timestamp Drift in ASR without Forgetting via Replay-Based Distribution Editing

83% match2026

Word Level Timestamp Generation for Automatic Speech Recognition and Translation

80% match1 repo17.8k2025

Gradient-Based Speech-to-Text Alignment for Any ASR Model: From CTC to Speech LLMs

78% match2026

Dynamic Alignment Mask CTC: Improved Mask-CTC with Aligned Cross Entropy

78% match2023

AliO: Output Alignment Matters in Long-Term Time Series Forecasing

78% match1 repo112026

Cite this paper

@article{shi2023achieving,
  title  = {Achieving Timestamp Prediction While Recognizing with Non-Autoregressive End-to-End ASR Model},
  author = {Xian Shi and Yanni Chen and Shiliang Zhang and Zhijie Yan},
  year   = {2023},
  eprint = {2301.12343},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2301.12343v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2301.12343.svg)](https://codeofpaper.com/paper/2301.12343)

Discussion