Front-End Adapter: Adapting Front-End Input of Speech based Self-Supervised Learning for Speech Recognition

Xie Chen, Ziyang Ma, Changli Tang, Yujin Wang, Zhisheng Zheng
2/18/2023
eess.AScs.CLcs.SD

No confident code implementation found for this paper. 2 possible matches can be shown below.

Abstract

Recent years have witnessed a boom in self-supervised learning (SSL) in various areas including speech processing. Speech based SSL models present promising performance in a range of speech related tasks. However, the training of SSL models is computationally expensive and a common practice is to fine-tune a released SSL model on the specific task. It is essential to use consistent front-end input during pre-training and fine-tuning. This consistency may introduce potential issues when the optimal front-end is not the same as that used in pre-training. In this paper, we propose a simple but effective front-end adapter to address this front-end discrepancy. By minimizing the distance between the outputs of different front-ends, the filterbank feature (Fbank) can be compatible with SSL models which are pre-trained with waveform. The experiment results demonstrate the effectiveness of our proposed front-end adapter on several popular SSL models for the speech recognition task.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. 2 weaker matches are hidden by default — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Self-supervised Adaptive Pre-training of Multilingual Speech Models for Language and Dialect Identification

87% match2023

Efficient Domain Adaptation for Speech Foundation Models

86% match2023

Supervised Acoustic Embeddings And Their Transferability Across Languages

85% match2023

Efficient Adapter Tuning of Pre-trained Speech Models for Automatic Speaker Verification

84% match2024

Replay to Remember: Continual Layer-Specific Fine-tuning for German Speech Recognition

84% match2023

Fast-HuBERT: An Efficient Training Framework for Self-Supervised Speech Representation Learning

83% match1 repo1002023

Cite this paper

@article{chen2023frontend,
  title  = {Front-End Adapter: Adapting Front-End Input of Speech based Self-Supervised Learning for Speech Recognition},
  author = {Xie Chen and Ziyang Ma and Changli Tang and Yujin Wang and Zhisheng Zheng},
  year   = {2023},
  eprint = {2302.09331},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2302.09331v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2302.09331.svg)](https://codeofpaper.com/paper/2302.09331)

Discussion