R-Spin: Efficient Speaker and Noise-invariant Representation Learning with Acoustic Pieces

Heng-Jui Chang, James Glass
11/15/2023
cs.CLcs.SDeess.AS

No official code found. Best implementation: vectominist/rspin (6 stars, MIT, last commit 2025-03-24).

Abstract

This paper introduces Robust Spin (R-Spin), a data-efficient domain-specific self-supervision method for speaker and noise-invariant speech representations by learning discrete acoustic units with speaker-invariant clustering (Spin). R-Spin resolves Spin's issues and enhances content representations by learning to predict acoustic pieces. R-Spin offers a 12X reduction in computational resources compared to previous state-of-the-art methods while outperforming them in severely distorted speech scenarios. This paper provides detailed analyses to show how discrete units contribute to speech encoder training and improving robustness in diverse acoustic environments.

View on arXivView PDF

Code Implementations(1)

vectominist/rspinCommunity67%
MIT

Official inference code for NAACL 2024 paper "R-Spin: Efficient Speaker and Noise-invariant Representation Learning with Acoustic Pieces"

61PythonNov 26, 20241 years agoMIT
audio-processingrobustnessself-supervised-learningspeech-processingspeech-recognition

Similar Papers

Self-supervised Fine-tuning for Improved Content Representations by Speaker-invariant Clustering

91% match1 repo662023

Analysing Discrete Self Supervised Speech Representation for Spoken Language Modeling

87% match1 repo212023

DinoSR: Self-Distillation and Online Clustering for Self-supervised Speech Representation Learning

86% match1 repo532026

GLaS-JEPA: Gaussian-Regularized Speech SSL without Engineered Prediction Targets

84% match2026

UNFUSED: UNsupervised Finetuning Using SElf supervised Distillation

83% match1 repo292023

An Information-Theoretic Analysis of Self-supervised Discrete Representations of Speech

83% match2023

Cite this paper

@article{chang2023rspin,
  title  = {R-Spin: Efficient Speaker and Noise-invariant Representation Learning with Acoustic Pieces},
  author = {Heng-Jui Chang and James Glass},
  year   = {2023},
  eprint = {2311.09117},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2311.09117v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2311.09117.svg)](https://codeofpaper.com/paper/2311.09117)

Discussion