Experimenting with Additive Margins for Contrastive Self-Supervised Speaker Verification

Theo Lepage, Reda Dehak
6/6/2023
eess.AScs.LG

No official code found. Best implementation: theolepage/sslsv (41 stars, MIT, PyTorch, last commit 2026-02-12).

Abstract

Most state-of-the-art self-supervised speaker verification systems rely on a contrastive-based objective function to learn speaker representations from unlabeled speech data. We explore different ways to improve the performance of these methods by: (1) revisiting how positive and negative pairs are sampled through a "symmetric" formulation of the contrastive loss; (2) introducing margins similar to AM-Softmax and AAM-Softmax that have been widely adopted in the supervised setting. We demonstrate the effectiveness of the symmetric contrastive loss which provides more supervision for the self-supervised task. Moreover, we show that Additive Margin and Additive Angular Margin allow reducing the overall number of false negatives and false positives by improving speaker separability. Finally, by combining both techniques and training a larger model we achieve 7.50% EER and 0.5804 minDCF on the VoxCeleb1 test set, which outperforms other contrastive self supervised methods on speaker verification.

View on arXivView PDF

Code Implementations(1)

theolepage/sslsvCommunity100%
PyTorchMIT

Toolkit for training and evaluating Self-Supervised Learning (SSL) frameworks for Speaker Verification (SV).

419Shell, PythonJun 23, 20228 months agoMIT
barlowtwinsbyolcpcdeep-learningdeepcluster+15 more

Similar Papers

Additive Margin in Contrastive Self-Supervised Frameworks to Learn Discriminative Speaker Representations

93% match1 repo412024

Label-Efficient Self-Supervised Speaker Verification With Information Maximization and Contrastive Learning

85% match1 repo412022

Self-Supervised Frameworks for Speaker Verification via Bootstrapped Positive Sampling

85% match1 repo412025

Clustering-based hard negative sampling for supervised contrastive speaker verification

83% match2025

SSPS: Self-Supervised Positive Sampling for Robust Self-Supervised Speaker Verification

82% match1 repo412025

Getting More for Less: Using Weak Labels and AV-Mixup for Robust Audio-Visual Speaker Verification

82% match2023

Cite this paper

@article{lepage2023experimenting,
  title  = {Experimenting with Additive Margins for Contrastive Self-Supervised Speaker Verification},
  author = {Theo Lepage and Reda Dehak},
  year   = {2023},
  eprint = {2306.03664},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2306.03664v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2306.03664.svg)](https://codeofpaper.com/paper/2306.03664)

Discussion