MIPS at SemEval-2024 Task 3: Multimodal Emotion-Cause Pair Extraction in Conversations with Multimodal Language Models

Zebang Cheng, Fuqiang Niu, Yuxiang Lin, Zhi-Qi Cheng, Bowen Zhang +1 more
3/31/2024
cs.CLcs.CVcs.MM

Official code: MIPS-COLT/MER-MCE (25 stars, last commit 2024-08-02).

Abstract

This paper presents our winning submission to Subtask 2 of SemEval 2024 Task 3 on multimodal emotion cause analysis in conversations. We propose a novel Multimodal Emotion Recognition and Multimodal Emotion Cause Extraction (MER-MCE) framework that integrates text, audio, and visual modalities using specialized emotion encoders. Our approach sets itself apart from top-performing teams by leveraging modality-specific features for enhanced emotion understanding and causality inference. Experimental evaluation demonstrates the advantages of our multimodal approach, with our submission achieving a competitive weighted F1 score of 0.3435, ranking third with a margin of only 0.0339 behind the 1st team and 0.0025 behind the 2nd team. Project: https://github.com/MIPS-COLT/MER-MCE.git

View on arXivView PDF

Code Implementations(1)

MIPS-COLT/MER-MCEOfficial100%

This paper presents our winning submission to Subtask 2 of SemEval 2024 Task 3 on multimodal emotion cause analysis in conversations.

253Shell, PythonMar 30, 20242 years ago

Similar Papers

SemEval-2024 Task 3: Multimodal Emotion Cause Analysis in Conversations

87% match2 repos272024

Benchmarking and Bridging Emotion Conflicts for Multimodal Emotion Reasoning

86% match1 repo422025

LastResort at SemEval-2024 Task 3: Exploring Multimodal Emotion Cause Pair Extraction as Sequence Labelling Task

86% match2024

Generative Emotion Cause Explanation in Multimodal Conversations

85% match2024

LyS at SemEval-2024 Task 3: An Early Prototype for End-to-End Multimodal Emotion Linking as Graph-Based Parsing

85% match2024

Learning What to Attend First: Modality-Importance-Guided Reasoning for Reliable Multimodal Emotion Understanding

85% match2025

Cite this paper

@article{cheng2024mips,
  title  = {MIPS at SemEval-2024 Task 3: Multimodal Emotion-Cause Pair Extraction in Conversations with Multimodal Language Models},
  author = {Zebang Cheng and Fuqiang Niu and Yuxiang Lin and Zhi-Qi Cheng and Bowen Zhang and Xiaojiang Peng},
  year   = {2024},
  eprint = {2404.00511},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2404.00511v3}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2404.00511.svg)](https://codeofpaper.com/paper/2404.00511)

Discussion