Fine-tuning wav2vec2 for speaker recognition

Nik Vaessen, David A. van Leeuwen
9/30/2021
cs.SDcs.LGeess.AS

Official code: nikvaessen/w2v2-speaker (144 stars, MIT, last commit 2022-05-10).

Abstract

This paper explores applying the wav2vec2 framework to speaker recognition instead of speech recognition. We study the effectiveness of the pre-trained weights on the speaker recognition task, and how to pool the wav2vec2 output sequence into a fixed-length speaker embedding. To adapt the framework to speaker recognition, we propose a single-utterance classification variant with CE or AAM softmax loss, and an utterance-pair classification variant with BCE loss. Our best performing variant, w2v2-aam, achieves a 1.88% EER on the extended voxceleb1 test set compared to 1.69% EER with an ECAPA-TDNN baseline. Code is available at https://github.com/nikvaessen/w2v2-speaker.

View on arXivView PDF

Code Implementations(1)

nikvaessen/w2v2-speakerOfficial100%
MIT

Research code for the paper "Fine-tuning wav2vec2 for speaker recognition" found at https://arxiv.org/abs/2109.15053

14414Shell, PythonSep 27, 20214 years agoMIT

Similar Papers

wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations

86% match1 repo32.2k2020

Unsupervised Speech Recognition

84% match1 repo32.2k2021

wav2vec: Unsupervised Pre-training for Speech Recognition

83% match1 repo32.2k2019

Speaker and Language Change Detection using Wav2vec2 and Whisper

83% match2023

Dialect Adaptation and Data Augmentation for Low-Resource ASR: TalTech Systems for the MADASR 2023 Challenge

83% match2023

A comparative analysis between Conformer-Transducer, Whisper, and wav2vec2 for improving the child speech recognition

81% match2023

Cite this paper

@article{vaessen2021finetuning,
  title  = {Fine-tuning wav2vec2 for speaker recognition},
  author = {Nik Vaessen and David A. van Leeuwen},
  year   = {2021},
  eprint = {2109.15053},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/2109.15053}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2109.15053.svg)](https://codeofpaper.com/paper/2109.15053)

Discussion