Mutilmodal Feature Extraction and Attention-based Fusion for Emotion Estimation in Videos

Tao Shu, Xinke Wang, Ruotong Wang, Chuang Chen, Yixin Zhang +1 more
3/18/2023
cs.CVcs.AI

No confident code implementation found for this paper. 1 possible match can be shown below.

Abstract

The continuous improvement of human-computer interaction technology makes it possible to compute emotions. In this paper, we introduce our submission to the CVPR 2023 Competition on Affective Behavior Analysis in-the-wild (ABAW). Sentiment analysis in human-computer interaction should, as far as possible Start with multiple dimensions, fill in the single imperfect emotion channel, and finally determine the emotion tendency by fitting multiple results. Therefore, We exploited multimodal features extracted from video of different lengths from the competition dataset, including audio, pose and images. Well-informed emotion representations drive us to propose a Attention-based multimodal framework for emotion estimation. Our system achieves the performance of 0.361 on the validation dataset. The code is available at [https://github.com/xkwangcn/ABAW-5th-RT-IAI].

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. 1 weaker match is hidden by default — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Feature-Based Dual Visual Feature Extraction Model for Compound Multimodal Emotion Recognition

88% match2025

Multimodal Feature Extraction and Fusion for Emotional Reaction Intensity Estimation and Expression Classification in Videos with Transformers

87% match2023

MIPS at SemEval-2024 Task 3: Multimodal Emotion-Cause Pair Extraction in Conversations with Multimodal Language Models

86% match1 repo252024

Omni-Emotion: Extending Video MLLM with Detailed Face and Audio Modeling for Multimodal Emotion Analysis

85% match2025

Interpretable Multimodal Emotion Recognition using Facial Features and Physiological Signals

85% match2023

Emotion-LLaMA: Multimodal Emotion Recognition and Reasoning with Instruction Tuning

85% match1 repo6172026

Cite this paper

@article{shu2023mutilmodal,
  title  = {Mutilmodal Feature Extraction and Attention-based Fusion for Emotion Estimation in Videos},
  author = {Tao Shu and Xinke Wang and Ruotong Wang and Chuang Chen and Yixin Zhang and Xiao Sun},
  year   = {2023},
  eprint = {2303.10421},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2303.10421v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2303.10421.svg)](https://codeofpaper.com/paper/2303.10421)

Discussion