Reconstruction as a Bridge for Event-Based Visual Question Answering

Hanyue Lou, Jiayi Zhou, Yang Zhang, Boyu Li, Yi Wang +2 more
12/12/2025
cs.CV

No official code found. Best implementation: HYLZ-2019/EvQA (3 stars, last commit 2026-01-28).

Abstract

Integrating event cameras with Multimodal Large Language Models (MLLMs) promises general scene understanding in challenging visual conditions, yet requires navigating a trade-off between preserving the unique advantages of event data and ensuring compatibility with frame-based models. We address this challenge by using reconstruction as a bridge, proposing a straightforward Frame-based Reconstruction and Tokenization (FRT) method and designing an efficient Adaptive Reconstruction and Tokenization (ART) method that leverages event sparsity. For robust evaluation, we introduce EvQA, the first objective, real-world benchmark for event-based MLLMs, comprising 1,000 event-Q&A pairs from 22 public datasets. Our experiments demonstrate that our methods achieve state-of-the-art performance on EvQA, highlighting the significant potential of MLLMs in event-based vision.

View on arXivView PDF

Code Implementations(1)

HYLZ-2019/EvQACommunity63%

Reconstruction as a Bridge for Event-Based Visual Question Answering

30CSS, HTMLDec 12, 20258 months ago

Similar Papers

Geometric-Photometric Event-based 3D Gaussian Ray Tracing

87% match1 repo242025

EventVL: Understand Event Streams via Multimodal Large Language Model

87% match2025

EvGGS: A Collaborative Learning Framework for Event-based Generalizable Gaussian Splatting

85% match1 repo372026

EventSTR: A Benchmark Dataset and Baselines for Event Stream based Scene Text Recognition

85% match1 repo72025

EventGPT: Event Stream Understanding with Multimodal Large Language Models

85% match1 repo1152026

Recent Event Camera Innovations: A Survey

85% match1 repo3262024

Cite this paper

@article{lou2025reconstruction,
  title  = {Reconstruction as a Bridge for Event-Based Visual Question Answering},
  author = {Hanyue Lou and Jiayi Zhou and Yang Zhang and Boyu Li and Yi Wang and Guangnan Ye and Boxin Shi},
  year   = {2025},
  eprint = {2512.11510},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2512.11510v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2512.11510.svg)](https://codeofpaper.com/paper/2512.11510)

Discussion