MultiClimate: Multimodal Stance Detection on Climate Change Videos

Jiawen Wang, Longfei Zuo, Siyao Peng, Barbara Plank
9/26/2024
cs.CLcs.CV

Official code: werywjw/MultiClimate (3 stars, CC0-1.0, last commit 2024-10-06).

Abstract

Climate change (CC) has attracted increasing attention in NLP in recent years. However, detecting the stance on CC in multimodal data is understudied and remains challenging due to a lack of reliable datasets. To improve the understanding of public opinions and communication strategies, this paper presents MultiClimate, the first open-source manually-annotated stance detection dataset with $100$ CC-related YouTube videos and $4,209$ frame-transcript pairs. We deploy state-of-the-art vision and language models, as well as multimodal models for MultiClimate stance detection. Results show that text-only BERT significantly outperforms image-only ResNet50 and ViT. Combining both modalities achieves state-of-the-art, $0.747$/$0.749$ in accuracy/F1. Our 100M-sized fusion models also beat CLIP and BLIP, as well as the much larger 9B-sized multimodal IDEFICS and text-only Llama3 and Gemma2, indicating that multimodal stance detection remains challenging for large language models. Our code, dataset, as well as supplementary materials, are available at https://github.com/werywjw/MultiClimate.

View on arXivView PDF

Code Implementations(1)

werywjw/MultiClimateOfficial100%
CC0-1.0

[EMNLP 2024 Workshop NLP4PI]🌏 MultiClimate: Multimodal Stance Detection on Climate Change Videos 🌎

31Python, Jupyter NotebookJun 11, 20242 years agoCC0-1.0
climate-changellmsmultimodalstance-detectionvlms

Similar Papers

Multimodal Multi-turn Conversation Stance Detection: A Challenge Dataset and Effective Model

84% match2024

CliME: Evaluating Multimodal Climate Discourse on Social Media and the Climate Alignment Quotient (CAQ)

83% match2025

Stance Detection on Social Media with Fine-Tuned Large Language Models

83% match2024

Large Language Models Meet Stance Detection: A Survey of Tasks, Methods, Applications, Challenges and Future Directions

82% match2025

Stance-Driven Multimodal Controlled Statement Generation: New Dataset and Task

81% match2025

Exploring Vision Language Models for Multimodal and Multilingual Stance Detection

81% match2025

Cite this paper

@article{wang2024multiclimate,
  title  = {MultiClimate: Multimodal Stance Detection on Climate Change Videos},
  author = {Jiawen Wang and Longfei Zuo and Siyao Peng and Barbara Plank},
  year   = {2024},
  eprint = {2409.18346},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2409.18346v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2409.18346.svg)](https://codeofpaper.com/paper/2409.18346)

Discussion