Acoustic Model Fusion for End-to-end Speech Recognition

Zhihong Lei, Mingbin Xu, Shiyi Han, Leo Liu, Zhen Huang +6 more
10/10/2023
cs.SDcs.LGeess.AS

No confident code implementation found for this paper.

Abstract

Recent advances in deep learning and automatic speech recognition (ASR) have enabled the end-to-end (E2E) ASR system and boosted the accuracy to a new level. The E2E systems implicitly model all conventional ASR components, such as the acoustic model (AM) and the language model (LM), in a single network trained on audio-text pairs. Despite this simpler system architecture, fusing a separate LM, trained exclusively on text corpora, into the E2E system has proven to be beneficial. However, the application of LM fusion presents certain drawbacks, such as its inability to address the domain mismatch issue inherent to the internal AM. Drawing inspiration from the concept of LM fusion, we propose the integration of an external AM into the E2E system to better address the domain mismatch. By implementing this novel approach, we have achieved a significant reduction in the word error rate, with an impressive drop of up to 14.3% across varied test sets. We also discovered that this AM fusion approach is particularly beneficial in enhancing named entity recognition.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. Any repos shown below are weak matches — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Delayed Fusion: Integrating Large Language Models into First-Pass Decoding in End-to-end Speech Recognition

85% match2025

Enhancing CTC-based speech recognition with diverse modeling units

84% match2024

A Deep Learning System for Domain-specific Speech Recognition

83% match2023

SepALM: Audio Language Models Are Error Correctors for Robust Speech Separation

81% match2025

Improving Domain-Specific ASR with LLM-Generated Contextual Descriptions

81% match2024

Better Pseudo-labeling with Multi-ASR Fusion and Error Correction by SpeechLLM

81% match2025

Cite this paper

@article{lei2023acoustic,
  title  = {Acoustic Model Fusion for End-to-end Speech Recognition},
  author = {Zhihong Lei and Mingbin Xu and Shiyi Han and Leo Liu and Zhen Huang and Tim Ng and Yuanyuan Zhang and Ernest Pusateri and Mirko Hannemann and Yaqiao Deng and Man-Hung Siu},
  year   = {2023},
  eprint = {2310.07062},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2310.07062v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2310.07062.svg)](https://codeofpaper.com/paper/2310.07062)

Discussion