Guided Attention for Next Active Object @ EGO4D STA Challenge

Sanket Thakur, Cigdem Beyan, Pietro Morerio, Vittorio Murino, Alessio Del Bue
5/25/2023
cs.CV

No official code found. Best implementation: sanketsans/ganov2 (9 stars, Apache-2.0, last commit 2024-10-14).

Abstract

In this technical report, we describe the Guided-Attention mechanism based solution for the short-term anticipation (STA) challenge for the EGO4D challenge. It combines the object detections, and the spatiotemporal features extracted from video clips, enhancing the motion and contextual information, and further decoding the object-centric and motion-centric information to address the problem of STA in egocentric videos. For the challenge, we build our model on top of StillFast with Guided Attention applied on fast network. Our model obtains better performance on the validation set and also achieves state-of-the-art (SOTA) results on the challenge test set for EGO4D Short-Term Object Interaction Anticipation Challenge.

View on arXivView PDF

Code Implementations(1)

sanketsans/ganov2Community100%
Apache-2.0

Winner of CVPR23 EGO4D STA challenge

91PythonJun 4, 20232 years agoApache-2.0
action-anticipationego4dvideo-understanding

Similar Papers

StillFast: An End-to-End Approach for Short-Term Object Interaction Anticipation

88% match1 repo132023

Enhancing Next Active Object-based Egocentric Action Anticipation with Guided Attention

87% match1 repo92023

Short-term Object Interaction Anticipation with Disentangled Object Detection @ Ego4D Short Term Object Interaction Anticipation Challenge

85% match1 repo12024

VISTA: Technical Report for the Ego4D Short-Term Object Interaction Anticipation at EgoVis 2026

85% match1 repo12026

Leveraging Gaze and Set-of-Mark in VLLMs for Human-Object Interaction Anticipation from Egocentric Videos

83% match2026

Streaming egocentric action anticipation: An evaluation scheme and approach

83% match2023

Cite this paper

@article{thakur2023guided,
  title  = {Guided Attention for Next Active Object @ EGO4D STA Challenge},
  author = {Sanket Thakur and Cigdem Beyan and Pietro Morerio and Vittorio Murino and Alessio Del Bue},
  year   = {2023},
  eprint = {2305.16066},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2305.16066v3}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2305.16066.svg)](https://codeofpaper.com/paper/2305.16066)

Discussion