Multimodal Emotion Recognition and Sentiment Analysis in Multi-Party Conversation Contexts

Aref Farhadipour, Hossein Ranjbar, Masoumeh Chapariniya, Teodora Vukovic, Sarah Ebling +1 more
3/9/2025
cs.CVcs.SDeess.AS

No official code found. Best implementation: HoseinRanjbar/Emotion_Recognition (14 stars, last commit 2024-09-12).

Abstract

Emotion recognition and sentiment analysis are pivotal tasks in speech and language processing, particularly in real-world scenarios involving multi-party, conversational data. This paper presents a multimodal approach to tackle these challenges on a well-known dataset. We propose a system that integrates four key modalities/channels using pre-trained models: RoBERTa for text, Wav2Vec2 for speech, a proposed FacialNet for facial expressions, and a CNN+Transformer architecture trained from scratch for video analysis. Feature embeddings from each modality are concatenated to form a multimodal vector, which is then used to predict emotion and sentiment labels. The multimodal system demonstrates superior performance compared to unimodal approaches, achieving an accuracy of 66.36% for emotion recognition and 72.15% for sentiment analysis.

View on arXivView PDF

Code Implementations(1)

HoseinRanjbar/Emotion_RecognitionCommunity61%

Future Technologies Conference 2025 - MULTIMODAL EMOTION RECOGNITION AND SENTIMENT ANALYSIS IN MULTI-PARTY CONVERSATION CONTEXTS

142Shell, PythonSep 5, 20242 years ago
computer-visionemotion-recognitionmeldsentiment-analysisvideo-processing

Similar Papers

Textualized and Feature-based Models for Compound Multimodal Emotion Recognition in the Wild

88% match2 repos92024

GCM-Net: Graph-enhanced Cross-Modal Infusion with a Metaheuristic-Driven Network for Video Sentiment and Emotion Analysis

87% match2024

Enhancing Sentiment Analysis through Multimodal Fusion: A BERT-DINOv2 Approach

87% match2025

Interpretable multimodal sentiment analysis based on textual modality descriptions by using large-scale language models

86% match1 repo42023

Dynamic Multimodal Sentiment Analysis: Leveraging Cross-Modal Attention for Enabled Classification

86% match2025

Rethinking Multimodal Sentiment Analysis: A High-Accuracy, Simplified Fusion Architecture

86% match2025

Cite this paper

@article{farhadipour2025multimodal,
  title  = {Multimodal Emotion Recognition and Sentiment Analysis in Multi-Party Conversation Contexts},
  author = {Aref Farhadipour and Hossein Ranjbar and Masoumeh Chapariniya and Teodora Vukovic and Sarah Ebling and Volker Dellwo},
  year   = {2025},
  eprint = {2503.06805},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2503.06805v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2503.06805.svg)](https://codeofpaper.com/paper/2503.06805)

Discussion