Navigating the Minefield of MT Beam Search in Cascaded Streaming Speech Translation

Rastislav Rabatin, Frank Seide, Ernie Chang
6/26/2024
cs.CLcs.LGeess.AS

No confident code implementation found for this paper.

Abstract

We adapt the well-known beam-search algorithm for machine translation to operate in a cascaded real-time speech translation system. This proved to be more complex than initially anticipated, due to four key challenges: (1) real-time processing of intermediate and final transcriptions with incomplete words from ASR, (2) emitting intermediate and final translations with minimal user perceived latency, (3) handling beam search hypotheses that have unequal length and different model state, and (4) handling sentence boundaries. Previous work in the field of simultaneous machine translation only implemented greedy decoding. We present a beam-search realization that handles all of the above, providing guidance through the minefield of challenges. Our approach increases the BLEU score by 1 point compared to greedy search, reduces the CPU time by up to 40% and character flicker rate by 20+% compared to a baseline heuristic that just retranslates input repeatedly.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. Any repos shown below are weak matches — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Overcoming Latency Bottlenecks in On-Device Speech Translation: A Cascaded Approach with Alignment-Based Streaming MT

86% match2025

When End-to-End is Overkill: Rethinking Cascaded Speech-to-Text Translation

81% match2025

Language-Informed Beam Search Decoding for Multilingual Machine Translation

80% match1 repo132024

CTC-GMM: CTC guided modality matching for fast and accurate streaming speech translation

80% match2024

EBBS: An Ensemble with Bi-Level Beam Search for Zero-Shot Machine Translation

79% match2024

GPU-Accelerated WFST Beam Search Decoder for CTC-based Speech Recognition

79% match1 repo912023

Cite this paper

@article{rabatin2024navigating,
  title  = {Navigating the Minefield of MT Beam Search in Cascaded Streaming Speech Translation},
  author = {Rastislav Rabatin and Frank Seide and Ernie Chang},
  year   = {2024},
  eprint = {2407.11010},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2407.11010v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2407.11010.svg)](https://codeofpaper.com/paper/2407.11010)

Discussion