LA-RAG:Enhancing LLM-based ASR Accuracy with Retrieval-Augmented Generation

Shaojun Li, Hengchao Shang, Daimeng Wei, Jiaxin Guo, Zongyao Li +3 more
9/13/2024
cs.SDcs.CLeess.AS

No confident code implementation found for this paper. 5 possible matches can be shown below.

Abstract

Recent advancements in integrating speech information into large language models (LLMs) have significantly improved automatic speech recognition (ASR) accuracy. However, existing methods often constrained by the capabilities of the speech encoders under varied acoustic conditions, such as accents. To address this, we propose LA-RAG, a novel Retrieval-Augmented Generation (RAG) paradigm for LLM-based ASR. LA-RAG leverages fine-grained token-level speech datastores and a speech-to-speech retrieval mechanism to enhance ASR accuracy via LLM in-context learning (ICL) capabilities. Experiments on Mandarin and various Chinese dialect datasets demonstrate significant improvements in ASR accuracy compared to existing methods, validating the effectiveness of our approach, especially in handling accent variations.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. 5 weaker matches are hidden by default — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

RAG-Boost: Retrieval-Augmented Generation Enhanced LLM-based Speech Recognition

89% match2025

Retrieval-Augmented Speech Recognition Approach for Domain Challenges

88% match2025

Unveiling the Potential of LLM-Based ASR on Chinese Open-Source Datasets

86% match1 repo152024

Bridging Speech and Text: Enhancing ASR with Pinyin-to-Character Pre-training in LLMs

86% match2024

Contextualization of ASR with LLM using phonetic retrieval-based augmentation

85% match2024

Retrieval Augmented Generation based context discovery for ASR

85% match2025

Cite this paper

@article{li2024laragenhancing,
  title  = {LA-RAG:Enhancing LLM-based ASR Accuracy with Retrieval-Augmented Generation},
  author = {Shaojun Li and Hengchao Shang and Daimeng Wei and Jiaxin Guo and Zongyao Li and Xianghui He and Min Zhang and Hao Yang},
  year   = {2024},
  eprint = {2409.08597},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2409.08597v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2409.08597.svg)](https://codeofpaper.com/paper/2409.08597)

Discussion