Improving Speech Emotion Recognition in Under-Resourced Languages via Speech-to-Speech Translation with Bootstrapping Data Selection

Hsi-Che Lin, Yi-Cheng Lin, Huang-Cheng Chou, Hung-yi Lee
9/17/2024
eess.AScs.CLcs.SD

No official code found. Best implementation: hsi-che-lin/Improve-SER-via-S2ST (3 stars, last commit 2025-01-07).

Abstract

Speech Emotion Recognition (SER) is a crucial component in developing general-purpose AI agents capable of natural human-computer interaction. However, building robust multilingual SER systems remains challenging due to the scarcity of labeled data in languages other than English and Chinese. In this paper, we propose an approach to enhance SER performance in low SER resource languages by leveraging data from high-resource languages. Specifically, we employ expressive Speech-to-Speech translation (S2ST) combined with a novel bootstrapping data selection pipeline to generate labeled data in the target language. Extensive experiments demonstrate that our method is both effective and generalizable across different upstream models and languages. Our results suggest that this approach can facilitate the development of more scalable and robust multilingual SER systems.

View on arXivView PDF

Code Implementations(1)

hsi-che-lin/Improve-SER-via-S2STCommunity100%
31Shell, PythonJan 7, 20251 years ago

Similar Papers

Learning More with Less: Self-Supervised Approaches for Low-Resource Speech Emotion Recognition

86% match2025

SER Evals: In-domain and Out-of-domain Benchmarking for Speech Emotion Recognition

85% match1 repo112024

Leveraging Cross-Attention Transformer and Multi-Feature Fusion for Cross-Linguistic Speech Emotion Recognition

84% match2025

Emotion Classification in Low and Moderate Resource Languages

84% match2024

A Layer-Anchoring Strategy for Enhancing Cross-Lingual Speech Emotion Recognition

84% match2024

MELD-ST: An Emotion-aware Speech Translation Dataset

84% match2024

Cite this paper

@article{lin2024improving,
  title  = {Improving Speech Emotion Recognition in Under-Resourced Languages via Speech-to-Speech Translation with Bootstrapping Data Selection},
  author = {Hsi-Che Lin and Yi-Cheng Lin and Huang-Cheng Chou and Hung-yi Lee},
  year   = {2024},
  eprint = {2409.10985},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2409.10985v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2409.10985.svg)](https://codeofpaper.com/paper/2409.10985)

Discussion