Twins-PainViT: Towards a Modality-Agnostic Vision Transformer Framework for Multimodal Automatic Pain Assessment using Facial Videos and fNIRS

Stefanos Gkikas, Manolis Tsiknakis
7/29/2024
cs.CVcs.AI

No confident code implementation found for this paper. 2 possible matches can be shown below.

Abstract

Automatic pain assessment plays a critical role for advancing healthcare and optimizing pain management strategies. This study has been submitted to the First Multimodal Sensing Grand Challenge for Next-Gen Pain Assessment (AI4PAIN). The proposed multimodal framework utilizes facial videos and fNIRS and presents a modality-agnostic approach, alleviating the need for domain-specific models. Employing a dual ViT configuration and adopting waveform representations for the fNIRS, as well as for the extracted embeddings from the two modalities, demonstrate the efficacy of the proposed method, achieving an accuracy of 46.76% in the multilevel pain assessment task.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. 2 weaker matches are hidden by default — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

PainFormer: a Vision Foundation Model for Automatic Pain Assessment

89% match1 repo52025

Synthetic Thermal and RGB Videos for Automatic Pain Assessment utilizing a Vision-MLP Architecture

85% match2024

Transformer with Leveraged Masked Autoencoder for video-based Pain Assessment

83% match2024

PainNet: Statistical Relation Network with Episode-Based Training for Pain Estimation

83% match2025

Advancing Multimodal Data Fusion in Pain Recognition: A Strategy Leveraging Statistical Correlation and Human-Centered Perspectives

83% match1 repo2024

Improving Pain Classification using Spatio-Temporal Deep Learning Approaches with Facial Expressions

82% match2025

Cite this paper

@article{gkikas2024twinspainvit,
  title  = {Twins-PainViT: Towards a Modality-Agnostic Vision Transformer Framework for Multimodal Automatic Pain Assessment using Facial Videos and fNIRS},
  author = {Stefanos Gkikas and Manolis Tsiknakis},
  year   = {2024},
  eprint = {2407.19809},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2407.19809v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2407.19809.svg)](https://codeofpaper.com/paper/2407.19809)

Discussion