Context and System Fusion in Post-ASR Emotion Recognition with Large Language Models

Pavel Stepachev, Pinzhen Chen, Barry Haddow
10/4/2024
cs.CLeess.AS

No official code found. Best implementation: rggdmonk/GenSEC-Task-3 (2 stars, Apache-2.0, last commit 2024-10-07).

Abstract

Large language models (LLMs) have started to play a vital role in modelling speech and text. To explore the best use of context and multiple systems' outputs for post-ASR speech emotion prediction, we study LLM prompting on a recent task named GenSEC. Our techniques include ASR transcript ranking, variable conversation context, and system output fusion. We show that the conversation context has diminishing returns and the metric used to select the transcript for prediction is crucial. Finally, our best submission surpasses the provided baseline by 20% in absolute accuracy.

View on arXivView PDF

Code Implementations(1)

rggdmonk/GenSEC-Task-3Community100%
Apache-2.0

Context and System Fusion in Post-ASR Emotion Recognition with Large Language Models | Submission to GenSEC Task 3 (LLM-Based Post ASR Speech Emotion Recognition Challenge)

20PythonOct 3, 20242 years agoApache-2.0
asr-transcribed-spoken-textasr-transcriptscontext-modellinggensecgensec-task-3+4 more

Similar Papers

Improving Speech-based Emotion Recognition with Contextual Utterance Analysis and LLMs

91% match2024

Revise, Reason, and Recognize: LLM-Based Emotion Recognition via Emotion-Specific Prompts and ASR Error Correction

87% match1 repo72024

How to Retrieve Examples in In-context Learning to Improve Conversational Emotion Recognition using Large Language Models?

85% match2025

Can Large Language Models Generate Effective Datasets for Emotion Recognition in Conversations?

85% match2025

EmoSLLM: Parameter-Efficient Adaptation of LLMs for Speech Emotion Recognition

84% match2025

Towards interfacing large language models with ASR systems using confidence measures and prompting

84% match2024

Cite this paper

@article{stepachev2024context,
  title  = {Context and System Fusion in Post-ASR Emotion Recognition with Large Language Models},
  author = {Pavel Stepachev and Pinzhen Chen and Barry Haddow},
  year   = {2024},
  eprint = {2410.03312},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2410.03312v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2410.03312.svg)](https://codeofpaper.com/paper/2410.03312)

Discussion