LASER: Learning by Aligning Self-supervised Representations of Speech for Improving Content-related Tasks

Amit Meghanani, Thomas Hain
6/13/2024
cs.CLcs.SDeess.AS

No official code found. Best implementation: Trikaldarshi/LASER (3 stars, last commit 2024-09-25).

Abstract

Self-supervised learning (SSL)-based speech models are extensively used for full-stack speech processing. However, it has been observed that improving SSL-based speech representations using unlabeled speech for content-related tasks is challenging and computationally expensive. Recent attempts have been made to address this issue with cost-effective self-supervised fine-tuning (SSFT) approaches. Continuing in this direction, a cost-effective SSFT method named "LASER: Learning by Aligning Self-supervised Representations" is presented. LASER is based on the soft-DTW alignment loss with temporal regularisation term. Experiments are conducted with HuBERT and WavLM models and evaluated on the SUPERB benchmark for two content-related tasks: automatic speech recognition (ASR) and phoneme recognition (PR). A relative improvement of 3.7% and 8.2% for HuBERT, and 4.1% and 11.7% for WavLM are observed, for the ASR and PR tasks respectively, with only < 3 hours of fine-tuning on a single GPU.

View on arXivView PDF

Code Implementations(1)

Trikaldarshi/LASERCommunity100%

LASER: Learning by Aligning Self-supervised Representations of Speech for Improving Content-related Tasks

30PythonJun 12, 20242 years ago
s3prlself-supervised-learning

Similar Papers

SCORE: Self-supervised Correspondence Fine-tuning for Improved Content Representations

88% match1 repo92024

CA-SSLR: Condition-Aware Self-Supervised Learning Representation for Generalized Speech Processing

83% match1 repo32024

Position-invariant Fine-tuning of Speech Enhancement Models with Self-supervised Speech Representations

83% match2026

Linear-Complexity Self-Supervised Learning for Speech Processing

82% match1 repo72024

Simultaneous or Sequential Training? How Speech Representations Cooperate in a Multi-Task Self-Supervised Learning System

81% match2023

How to Learn a New Language? An Efficient Solution for Self-Supervised Learning Models Unseen Languages Adaption in Low-Resource Scenario

81% match2024

Cite this paper

@article{meghanani2024laser,
  title  = {LASER: Learning by Aligning Self-supervised Representations of Speech for Improving Content-related Tasks},
  author = {Amit Meghanani and Thomas Hain},
  year   = {2024},
  eprint = {2406.09153},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2406.09153v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2406.09153.svg)](https://codeofpaper.com/paper/2406.09153)

Discussion