ObjectNLQ @ Ego4D Episodic Memory Challenge 2024

Yisen Feng, Haoyu Zhang, Yuquan Xie, Zaijing Li, Meng Liu +1 more
6/22/2024
cs.CVcs.AI

Official code: Yisen-Feng/ObjectNLQ (1 star, MIT, last commit 2024-07-11).

Abstract

In this report, we present our approach for the Natural Language Query track and Goal Step track of the Ego4D Episodic Memory Benchmark at CVPR 2024. Both challenges require the localization of actions within long video sequences using textual queries. To enhance localization accuracy, our method not only processes the temporal information of videos but also identifies fine-grained objects spatially within the frames. To this end, we introduce a novel approach, termed ObjectNLQ, which incorporates an object branch to augment the video representation with detailed object information, thereby improving grounding efficiency. ObjectNLQ achieves a mean R@1 of 23.15, ranking 2nd in the Natural Language Queries Challenge, and gains 33.00 in terms of the metric R@1, IoU=0.3, ranking 3rd in the Goal Step Challenge. Our code will be released at https://github.com/Yisen-Feng/ObjectNLQ.

View on arXivView PDF

Code Implementations(1)

Yisen-Feng/ObjectNLQOfficial100%
MIT
10C++, ShellJul 11, 20242 years agoMIT

Similar Papers

Relocate: A Simple Training-Free Baseline for Visual Query Localization Using Region-Based Representations

87% match1 repo2026

OmniSTVG: Toward Spatio-Temporal Omni-Object Video Grounding

86% match1 repo112025

TimeLoc: A Unified End-to-End Framework for Precise Timestamp Localization in Long Videos

85% match1 repo102025

SVAG-Bench: A Large-Scale Benchmark for Multi-Instance Spatio-temporal Video Action Grounding

84% match2025

Localizing Events in Videos with Multimodal Queries

84% match2026

Open-Vocabulary Action Localization with Iterative Visual Prompting

84% match2024

Cite this paper

@article{feng2024objectnlq,
  title  = {ObjectNLQ @ Ego4D Episodic Memory Challenge 2024},
  author = {Yisen Feng and Haoyu Zhang and Yuquan Xie and Zaijing Li and Meng Liu and Liqiang Nie},
  year   = {2024},
  eprint = {2406.15778},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2406.15778v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2406.15778.svg)](https://codeofpaper.com/paper/2406.15778)

Discussion