Turning Whisper into Real-Time Transcription System

Dominik Macháček, Raj Dabre, Ondřej Bojar
7/27/2023
cs.CL

No official code found. Best implementation: ufal/whisper_streaming (3,671 stars, MIT, last commit 2025-11-12).

Abstract

Whisper is one of the recent state-of-the-art multilingual speech recognition and translation models, however, it is not designed for real time transcription. In this paper, we build on top of Whisper and create Whisper-Streaming, an implementation of real-time speech transcription and translation of Whisper-like models. Whisper-Streaming uses local agreement policy with self-adaptive latency to enable streaming transcription. We show that Whisper-Streaming achieves high quality and 3.3 seconds latency on unsegmented long-form speech transcription test set, and we demonstrate its robustness and practical usability as a component in live transcription service at a multilingual conference.

View on arXivView PDF

Code Implementations(1)

ufal/whisper_streamingCommunity100%
MIT

Whisper realtime streaming for long speech-to-text transcription and translation

3,671407PythonApr 5, 202311 months agoMIT

Similar Papers

Leveraging Timestamp Information for Serialized Joint Streaming Recognition and Translation

85% match2023

Token-Level Serialized Output Training for Joint Streaming ASR and ST Leveraging Textual Alignments

85% match2023

Speech Translation with Large Language Models: An Industrial Practice

84% match2023

Seamless: Multilingual Expressive and Streaming Speech Translation

84% match1 repo11.9k2023

Streaming Translation and Transcription Through Speech-to-Text Causal Alignment

84% match2026

Overcoming Latency Bottlenecks in On-Device Speech Translation: A Cascaded Approach with Alignment-Based Streaming MT

83% match2025

Cite this paper

@article{machek2023turning,
  title  = {Turning Whisper into Real-Time Transcription System},
  author = {Dominik Macháček and Raj Dabre and Ondřej Bojar},
  year   = {2023},
  eprint = {2307.14743},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2307.14743v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2307.14743.svg)](https://codeofpaper.com/paper/2307.14743)

Discussion