ESPnet-EZ: Python-only ESPnet for Easy Fine-tuning and Integration

Masao Someki, Kwanghee Choi, Siddhant Arora, William Chen, Samuele Cornell +5 more
9/14/2024
cs.SDcs.AIeess.AS

No confident code implementation found for this paper.

Abstract

We introduce ESPnet-EZ, an extension of the open-source speech processing toolkit ESPnet, aimed at quick and easy development of speech models. ESPnet-EZ focuses on two major aspects: (i) easy fine-tuning and inference of existing ESPnet models on various tasks and (ii) easy integration with popular deep neural network frameworks such as PyTorch-Lightning, Hugging Face transformers and datasets, and Lhotse. By replacing ESPnet design choices inherited from Kaldi with a Python-only, Bash-free interface, we dramatically reduce the effort required to build, debug, and use a new model. For example, to fine-tune a speech foundation model, ESPnet-EZ, compared to ESPnet, reduces the number of newly written code by 2.7x and the amount of dependent code by 6.7x while dramatically reducing the Bash script dependencies. The codebase of ESPnet-EZ is publicly available.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. Any repos shown below are weak matches — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

The PyTorch-Kaldi Speech Recognition Toolkit

84% match1 repo2.4k2018

SpeechBrain: A General-Purpose Speech Toolkit

84% match1 repo11.8k2021

Encoder-Side Neuron Identification and Amplification for Acoustic Perception in Large Audio-Language Models

83% match2026

Towards Improving the Performance of Pre-Trained Speech Models for Low-Resource Languages Through Lateral Inhibition

82% match2023

Self-Guidance: Enhancing Neural Codecs via Decoder Manifold Alignment

82% match2026

EffiFusion-GAN: Efficient Fusion Generative Adversarial Network for Speech Enhancement

82% match2025

Cite this paper

@article{someki2024espnetez,
  title  = {ESPnet-EZ: Python-only ESPnet for Easy Fine-tuning and Integration},
  author = {Masao Someki and Kwanghee Choi and Siddhant Arora and William Chen and Samuele Cornell and Jionghao Han and Yifan Peng and Jiatong Shi and Vaibhav Srivastav and Shinji Watanabe},
  year   = {2024},
  eprint = {2409.09506},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2409.09506v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2409.09506.svg)](https://codeofpaper.com/paper/2409.09506)

Discussion