Self-supervised Fine-tuning for Improved Content Representations by Speaker-invariant Clustering

Heng-Jui Chang, Alexander H. Liu, James Glass
5/18/2023
cs.CLeess.AS

No official code found. Best implementation: vectominist/spin (66 stars, MIT, last commit 2023-05-19).

Abstract

Self-supervised speech representation models have succeeded in various tasks, but improving them for content-related problems using unlabeled data is challenging. We propose speaker-invariant clustering (Spin), a novel self-supervised learning method that clusters speech representations and performs swapped prediction between the original and speaker-perturbed utterances. Spin disentangles speaker information and preserves content representations with just 45 minutes of fine-tuning on a single GPU. Spin improves pre-trained networks and outperforms prior methods in speech recognition and acoustic unit discovery.

View on arXivView PDF

Code Implementations(1)

vectominist/spinCommunity100%
MIT

Official code for Interspeech 2023 paper "Self-supervised Fine-tuning for Improved Content Representations by Speaker-invariant Clustering"

667Shell, PythonMay 17, 20233 years agoMIT
clusteringdisentanglementself-supervised-learningspeech-processingspeech-recognition

Similar Papers

R-Spin: Efficient Speaker and Noise-invariant Representation Learning with Acoustic Pieces

91% match1 repo62023

DinoSR: Self-Distillation and Online Clustering for Self-supervised Speech Representation Learning

86% match1 repo532026

UNFUSED: UNsupervised Finetuning Using SElf supervised Distillation

84% match1 repo292023

Speaker Group Encoding in Self-supervised Speech Recognition Models

84% match2026

MooseNet: A Trainable Metric for Synthesized Speech with a PLDA Module

84% match1 repo62023

Replay to Remember: Continual Layer-Specific Fine-tuning for German Speech Recognition

84% match2023

Cite this paper

@article{chang2023selfsupervised,
  title  = {Self-supervised Fine-tuning for Improved Content Representations by Speaker-invariant Clustering},
  author = {Heng-Jui Chang and Alexander H. Liu and James Glass},
  year   = {2023},
  eprint = {2305.11072},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2305.11072v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2305.11072.svg)](https://codeofpaper.com/paper/2305.11072)

Discussion