Automatic Speech Recognition with BERT and CTC Transformers: A Review

Noussaiba Djeffal, Hamza Kheddar, Djamel Addou, Ahmed Cherif Mazari, Yassine Himeur
10/12/2024
cs.CLeess.AS

No confident code implementation found for this paper.

Abstract

This review paper provides a comprehensive analysis of recent advances in automatic speech recognition (ASR) with bidirectional encoder representations from transformers BERT and connectionist temporal classification (CTC) transformers. The paper first introduces the fundamental concepts of ASR and discusses the challenges associated with it. It then explains the architecture of BERT and CTC transformers and their potential applications in ASR. The paper reviews several studies that have used these models for speech recognition tasks and discusses the results obtained. Additionally, the paper highlights the limitations of these models and outlines potential areas for further research. All in all, this review provides valuable insights for researchers and practitioners who are interested in ASR with BERT and CTC transformers.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. Any repos shown below are weak matches — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Cascaded Cross-Modal Transformer for Audio-Textual Classification

80% match1 repo32024

Bypass Temporal Classification: Weakly Supervised Automatic Speech Recognition with Imperfect Transcripts

80% match2023

Speech Recognition Transformers: Topological-lingualism Perspective

79% match2024

UniEnc-CASSNAT: An Encoder-only Non-autoregressive ASR for Speech SSL Models

79% match2024

Guiding Frame-Level CTC Alignments Using Self-knowledge Distillation

79% match1 repo72024

The evaluation of a code-switched Sepedi-English automatic speech recognition system

78% match2024

Cite this paper

@article{djeffal2024automatic,
  title  = {Automatic Speech Recognition with BERT and CTC Transformers: A Review},
  author = {Noussaiba Djeffal and Hamza Kheddar and Djamel Addou and Ahmed Cherif Mazari and Yassine Himeur},
  year   = {2024},
  eprint = {2410.09456},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2410.09456v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2410.09456.svg)](https://codeofpaper.com/paper/2410.09456)

Discussion