Look and Tell: A Dataset for Multimodal Grounding Across Egocentric and Exocentric Views

Anna Deichler, Jonas Beskow
10/26/2025
cs.CVcs.CLcs.RO

No confident code implementation found for this paper.

Abstract

We introduce Look and Tell, a multimodal dataset for studying referential communication across egocentric and exocentric perspectives. Using Meta Project Aria smart glasses and stationary cameras, we recorded synchronized gaze, speech, and video as 25 participants instructed a partner to identify ingredients in a kitchen. Combined with 3D scene reconstructions, this setup provides a benchmark for evaluating how different spatial representations (2D vs. 3D; ego vs. exo) affect multimodal grounding. The dataset contains 3.67 hours of recordings, including 2,707 richly annotated referential expressions, and is designed to advance the development of embodied agents that can understand and engage in situated dialogue.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. Any repos shown below are weak matches — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Grounding Language in Multi-Perspective Referential Communication

85% match1 repo22024

EmbodiedScan: A Holistic Multi-Modal 3D Perception Suite Towards Embodied AI

82% match1 repo6712026

ChatterBox: Multi-round Multimodal Referring and Grounding

81% match1 repo622024

Does Visual Grounding Enhance the Understanding of Embodied Knowledge in Large Language Models?

80% match1 repo32025

IndEgo: A Dataset of Industrial Scenarios and Collaborative Work for Egocentric Assistants

80% match1 repo92025

Visual Grounding from Event Cameras

80% match2025

Cite this paper

@article{deichler2025look,
  title  = {Look and Tell: A Dataset for Multimodal Grounding Across Egocentric and Exocentric Views},
  author = {Anna Deichler and Jonas Beskow},
  year   = {2025},
  eprint = {2510.22672},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2510.22672v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2510.22672.svg)](https://codeofpaper.com/paper/2510.22672)

Discussion