MAGIC-TBR: Multiview Attention Fusion for Transformer-based Bodily Behavior Recognition in Group Settings

Surbhi Madan, Rishabh Jain, Gulshan Sharma, Ramanathan Subramanian, Abhinav Dhall
9/19/2023
cs.CVcs.HCcs.MM

Official code: surbhimadan92/MAGIC-TBR (3 stars, last commit 2023-08-23).

Abstract

Bodily behavioral language is an important social cue, and its automated analysis helps in enhancing the understanding of artificial intelligence systems. Furthermore, behavioral language cues are essential for active engagement in social agent-based user interactions. Despite the progress made in computer vision for tasks like head and body pose estimation, there is still a need to explore the detection of finer behaviors such as gesturing, grooming, or fumbling. This paper proposes a multiview attention fusion method named MAGIC-TBR that combines features extracted from videos and their corresponding Discrete Cosine Transform coefficients via a transformer-based approach. The experiments are conducted on the BBSI dataset and the results demonstrate the effectiveness of the proposed feature fusion with multiview attention. The code is available at: https://github.com/surbhimadan92/MAGIC-TBR

View on arXivView PDF

Code Implementations(1)

surbhimadan92/MAGIC-TBROfficial100%
31Python, Jupyter NotebookJul 27, 20233 years ago

Similar Papers

GazeVLM: A Vision-Language Model for Multi-Task Gaze Understanding

86% match2025

ViTGaze: Gaze Following with Interaction Features in Vision Transformers

84% match1 repo622024

Summarize the Past to Predict the Future: Natural Language Descriptions of Context Boost Multimodal Object Interaction Anticipation

83% match1 repo82023

Multi-view Action Recognition via Directed Gromov-Wasserstein Discrepancy

83% match2024

Human-Object Interaction Prediction in Videos through Gaze Following

83% match1 repo312023

LS-HAR: Language Supervised Human Action Recognition with Salient Fusion, Construction Sites as a Use-Case

83% match2024

Cite this paper

@article{madan2023magictbr,
  title  = {MAGIC-TBR: Multiview Attention Fusion for Transformer-based Bodily Behavior Recognition in Group Settings},
  author = {Surbhi Madan and Rishabh Jain and Gulshan Sharma and Ramanathan Subramanian and Abhinav Dhall},
  year   = {2023},
  eprint = {2309.10765},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2309.10765v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2309.10765.svg)](https://codeofpaper.com/paper/2309.10765)

Discussion