A vector quantized masked autoencoder for speech emotion recognition

Samir Sadok, Simon Leglaive, Renaud Séguier
4/21/2023
cs.SDcs.LGeess.AS

No official code found. Best implementation: samsad35/VQ-MAE-S-code (30 stars, AGPL-3.0, last commit 2024-03-04).

Abstract

Recent years have seen remarkable progress in speech emotion recognition (SER), thanks to advances in deep learning techniques. However, the limited availability of labeled data remains a significant challenge in the field. Self-supervised learning has recently emerged as a promising solution to address this challenge. In this paper, we propose the vector quantized masked autoencoder for speech (VQ-MAE-S), a self-supervised model that is fine-tuned to recognize emotions from speech signals. The VQ-MAE-S model is based on a masked autoencoder (MAE) that operates in the discrete latent space of a vector-quantized variational autoencoder. Experimental results show that the proposed VQ-MAE-S model, pre-trained on the VoxCeleb2 dataset and fine-tuned on emotional speech data, outperforms an MAE working on the raw spectrogram representation and other state-of-the-art methods in SER.

View on arXivView PDF

Code Implementations(1)

samsad35/VQ-MAE-S-codeCommunity100%
AGPL-3.0

[ICASSPW] A Vector Quantized Masked AutoEncoder for speech emotion recognition

301PythonOct 24, 20232 years agoAGPL-3.0
emotion-recognitionmasked-autoencoderself-supervised-learningspeechvector-quantization

Similar Papers

A vector quantized masked autoencoder for audiovisual speech emotion recognition

94% match1 repo42023

Self-Supervised Learning for Audio-Based Emotion Recognition

83% match1 repo1212023

CochCeps-Augment: A Novel Self-Supervised Contrastive Learning Using Cochlear Cepstrum-based Masking for Speech Emotion Recognition

82% match2024

Unsupervised Representations Improve Supervised Learning in Speech Emotion Recognition

82% match2023

Amplifying Emotional Signals: Data-Efficient Deep Learning for Robust Speech Emotion Recognition

81% match2025

Exploring Self-Supervised Multi-view Contrastive Learning for Speech Emotion Recognition with Limited Annotations

81% match2024

Cite this paper

@article{sadok2023vector,
  title  = {A vector quantized masked autoencoder for speech emotion recognition},
  author = {Samir Sadok and Simon Leglaive and Renaud Séguier},
  year   = {2023},
  eprint = {2304.11117},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2304.11117v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2304.11117.svg)](https://codeofpaper.com/paper/2304.11117)

Discussion