InternVideo2.5: Empowering Video MLLMs with Long and Rich Context Modeling

Yi Wang, Xinhao Li, Ziang Yan, Yinan He, Jiashuo Yu +11 more
1/21/2025
cs.CV

Official code: OpenGVLab/InternVideo (2,343 stars, Apache-2.0, last commit 2026-03-25).

Abstract

This paper aims to improve the performance of video multimodal large language models (MLLM) via long and rich context (LRC) modeling. As a result, we develop a new version of InternVideo2.5 with a focus on enhancing the original MLLMs' ability to perceive fine-grained details and capture long-form temporal structure in videos. Specifically, our approach incorporates dense vision task annotations into MLLMs using direct preference optimization and develops compact spatiotemporal representations through adaptive hierarchical token compression. Experimental results demonstrate this unique design of LRC greatly improves the results of video MLLM in mainstream video understanding benchmarks (short & long), enabling the MLLM to memorize significantly longer video inputs (at least 6x longer than the original), and master specialized vision capabilities like object tracking and segmentation. Our work highlights the importance of multimodal context richness (length and fineness) in empowering MLLM's innate abilites (focus and memory), providing new insights for future research on video MLLM. Code and models are available at https://github.com/OpenGVLab/InternVideo/tree/main/InternVideo2.5

View on arXivView PDF

Code Implementations(1)

OpenGVLab/InternVideoOfficial100%
Apache-2.0

[ECCV2024] Video Foundation Models & Data for Multimodal Understanding

2,343158C, C++Nov 23, 20226 months agoApache-2.0
action-recognitionbenchmarkcontrastive-learningfoundation-modelsinstruction-tuning+15 more

Similar Papers

TimeMarker: A Versatile Video-LLM for Long and Short Video Understanding with Superior Temporal Localization Ability

90% match1 repo1092024

Long Context Transfer from Language to Vision

90% match1 repo4132024

Multimodal Instruction Tuning with Hybrid State Space Models

90% match2024

HierarQ: Task-Aware Hierarchical Q-Former for Enhanced Video Understanding

90% match1 repo22026

Seeing the Forest and the Trees: Query-Aware Tokenizer for Long-Video Multimodal Language Models

89% match2025

LVBench: An Extreme Long Video Understanding Benchmark

89% match1 repo1502024

Cite this paper

@article{wang2025internvideo,
  title  = {InternVideo2.5: Empowering Video MLLMs with Long and Rich Context Modeling},
  author = {Yi Wang and Xinhao Li and Ziang Yan and Yinan He and Jiashuo Yu and Xiangyu Zeng and Chenting Wang and Changlian Ma and Haian Huang and Jianfei Gao and Min Dou and Kai Chen and Wenhai Wang and Yu Qiao and Yali Wang and Limin Wang},
  year   = {2025},
  eprint = {2501.12386},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2501.12386v3}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2501.12386.svg)](https://codeofpaper.com/paper/2501.12386)

Discussion