Hierarchical Memory for Long Video QA

Yiqin Wang, Haoji Zhang, Yansong Tang, Yong Liu, Jiashi Feng +2 more
6/30/2024
cs.CV

No confident code implementation found for this paper. 3 possible matches can be shown below.

Abstract

This paper describes our champion solution to the LOVEU Challenge @ CVPR'24, Track 1 (Long Video VQA). Processing long sequences of visual tokens is computationally expensive and memory-intensive, making long video question-answering a challenging task. The key is to compress visual tokens effectively, reducing memory footprint and decoding latency, while preserving the essential information for accurate question-answering. We adopt a hierarchical memory mechanism named STAR Memory, proposed in Flash-VStream, that is capable of processing long videos with limited GPU memory (VRAM). We further utilize the video and audio data of MovieChat-1K training set to fine-tune the pretrained weight released by Flash-VStream, achieving 1st place in the challenge. Code is available at project homepage https://invinciblewyq.github.io/vstream-page .

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. 3 weaker matches are hidden by default — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

ReWind: Understanding Long Videos with Instructed Learnable Memory

88% match2026

Streaming Long Video Understanding with Large Language Models

86% match2026

StreamMem: Query-Agnostic KV Cache Memory for Streaming Video Understanding

86% match2025

CacheFlow: Compressive Streaming Memory for Efficient Long-Form Video Understanding

86% match2025

∞-Video: A Training-Free Approach to Long Video Understanding via Continuous-Time Memory Consolidation

86% match1 repo222026

DynFocus: Dynamic Cooperative Network Empowers LLMs with Video Understanding

86% match1 repo102026

Cite this paper

@article{wang2024hierarchical,
  title  = {Hierarchical Memory for Long Video QA},
  author = {Yiqin Wang and Haoji Zhang and Yansong Tang and Yong Liu and Jiashi Feng and Jifeng Dai and Xiaojie Jin},
  year   = {2024},
  eprint = {2407.00603},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2407.00603v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2407.00603.svg)](https://codeofpaper.com/paper/2407.00603)

Discussion