A Comparative Study on E-Branchformer vs Conformer in Speech Recognition, Translation, and Understanding Tasks

Yifan Peng, Kwangyoun Kim, Felix Wu, Brian Yan, Siddhant Arora +5 more
5/18/2023
cs.CLcs.SDeess.AS

No official code found. Best implementation: espnet/espnet (9,991 stars, Apache-2.0, PyTorch, last commit 2026-04-27).

Abstract

Conformer, a convolution-augmented Transformer variant, has become the de facto encoder architecture for speech processing due to its superior performance in various tasks, including automatic speech recognition (ASR), speech translation (ST) and spoken language understanding (SLU). Recently, a new encoder called E-Branchformer has outperformed Conformer in the LibriSpeech ASR benchmark, making it promising for more general speech applications. This work compares E-Branchformer and Conformer through extensive experiments using different types of end-to-end sequence-to-sequence models. Results demonstrate that E-Branchformer achieves comparable or better performance than Conformer in almost all evaluation sets across 15 ASR, 2 ST, and 3 SLU benchmarks, while being more stable during training. We will release our training configurations and pre-trained models for reproducibility, which can benefit the speech community.

View on arXivView PDF

Code Implementations(1)

espnet/espnetCommunity100%
PyTorchApache-2.0

End-to-End Speech Processing Toolkit

9,9912,444C++, CudaDec 13, 20175 months agoApache-2.0
chainerdeep-learningend-to-endkaldimachine-translation+11 more

Similar Papers

Investigating Pre-trained Audio Encoders in the Low-Resource Condition

83% match1 repo52023

Towards Improving the Performance of Pre-Trained Speech Models for Low-Resource Languages Through Lateral Inhibition

83% match2023

Improving Mandarin Speech Recogntion with Block-augmented Transformer

82% match2 repos142022

I3D: Transformer architectures with input-dependent dynamic depth for speech recognition

82% match2023

SALTTS: Leveraging Self-Supervised Speech Representations for improved Text-to-Speech Synthesis

81% match2023

Encoder-Side Neuron Identification and Amplification for Acoustic Perception in Large Audio-Language Models

81% match2026

Cite this paper

@article{peng2023comparative,
  title  = {A Comparative Study on E-Branchformer vs Conformer in Speech Recognition, Translation, and Understanding Tasks},
  author = {Yifan Peng and Kwangyoun Kim and Felix Wu and Brian Yan and Siddhant Arora and William Chen and Jiyang Tang and Suwon Shon and Prashant Sridhar and Shinji Watanabe},
  year   = {2023},
  eprint = {2305.11073},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2305.11073v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2305.11073.svg)](https://codeofpaper.com/paper/2305.11073)

Discussion