Object State Change Classification in Egocentric Videos using the Divided Space-Time Attention Mechanism

Md Mohaiminul Islam, Gedas Bertasius
7/24/2022
cs.CV

Official code: md-mohaiminul/ObjectStateChange (4 stars, last commit 2022-07-26).

Abstract

This report describes our submission called "TarHeels" for the Ego4D: Object State Change Classification Challenge. We use a transformer-based video recognition model and leverage the Divided Space-Time Attention mechanism for classifying object state change in egocentric videos. Our submission achieves the second-best performance in the challenge. Furthermore, we perform an ablation study to show that identifying object state change in egocentric videos requires temporal modeling ability. Lastly, we present several positive and negative examples to visualize our model's predictions. The code is publicly available at: https://github.com/md-mohaiminul/ObjectStateChange

View on arXivView PDF

Code Implementations(1)

md-mohaiminul/ObjectStateChangeOfficial100%
41Shell, PythonJul 24, 20224 years agoNOASSERTION

Similar Papers

EgoViT: Pyramid Video Transformer for Egocentric Action Recognition

87% match2023

Exploring Missing Modality in Multimodal Egocentric Datasets

85% match2024

Short-term Object Interaction Anticipation with Disentangled Object Detection @ Ego4D Short Term Object Interaction Anticipation Challenge

85% match1 repo12024

Multimodal Distillation for Egocentric Action Recognition

84% match1 repo322023

Self-Supervised Object Detection from Egocentric Videos

84% match2023

Action Scene Graphs for Long-Form Understanding of Egocentric Videos

84% match1 repo502023

Cite this paper

@article{islam2022object,
  title  = {Object State Change Classification in Egocentric Videos using the Divided Space-Time Attention Mechanism},
  author = {Md Mohaiminul Islam and Gedas Bertasius},
  year   = {2022},
  eprint = {2207.11814},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2207.11814v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2207.11814.svg)](https://codeofpaper.com/paper/2207.11814)

Discussion