MAVEN: Multi-modal Attention for Valence-Arousal Emotion Network

Vrushank Ahire, Kunal Shah, Mudasir Nazir Khan, Nikhil Pakhale, Lownish Rai Sookha +2 more
3/16/2025
cs.LGcs.AIcs.CV

Official code: Vrushank-Ahire/MAVEN_8th_ABAW (15 stars, MIT, last commit 2025-06-13).

Abstract

Dynamic emotion recognition in the wild remains challenging due to the transient nature of emotional expressions and temporal misalignment of multi-modal cues. Traditional approaches predict valence and arousal and often overlook the inherent correlation between these two dimensions. The proposed Multi-modal Attention for Valence-Arousal Emotion Network (MAVEN) integrates visual, audio, and textual modalities through a bi-directional cross-modal attention mechanism. MAVEN uses modality-specific encoders to extract features from synchronized video frames, audio segments, and transcripts, predicting emotions in polar coordinates following Russell's circumplex model. The evaluation of the Aff-Wild2 dataset using MAVEN achieved a concordance correlation coefficient (CCC) of 0.3061, surpassing the ResNet-50 baseline model with a CCC of 0.22. The multistage architecture captures the subtle and transient nature of emotional expressions in conversational videos and improves emotion recognition in real-world situations. The code is available at: https://github.com/Vrushank-Ahire/MAVEN_8th_ABAW

View on arXivView PDF

Code Implementations(1)

Vrushank-Ahire/MAVEN_8th_ABAWOfficial100%
MIT

MAVEN: Multi-modal Attention for Valence-Arousal Emotion Network

154PythonMar 16, 20251 years agoMIT

Similar Papers

Multimodal Emotion Recognition via Bi-directional Cross-Attention and Temporal Modeling

86% match2026

Recursive Joint Cross-Modal Attention for Multimodal Fusion in Dimensional Emotion Recognition

85% match1 repo322024

Team RAS in 10th ABAW Competition: Multimodal Valence and Arousal Estimation Approach

85% match2026

SUN Team's Contribution to ABAW 2024 Competition: Audio-visual Valence-Arousal Estimation and Expression Recognition

85% match2024

GIA-MIC: Multimodal Emotion Recognition with Gated Interactive Attention and Modality-Invariant Learning Constraints

84% match2025

AMuSE: Adaptive Multimodal Analysis for Speaker Emotion Recognition in Group Conversations

84% match2024

Cite this paper

@article{ahire2025maven,
  title  = {MAVEN: Multi-modal Attention for Valence-Arousal Emotion Network},
  author = {Vrushank Ahire and Kunal Shah and Mudasir Nazir Khan and Nikhil Pakhale and Lownish Rai Sookha and M. A. Ganaie and Abhinav Dhall},
  year   = {2025},
  eprint = {2503.12623},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2503.12623v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2503.12623.svg)](https://codeofpaper.com/paper/2503.12623)

Discussion