Understanding 3D Object Interaction from a Single Image

Shengyi Qian, David F. Fouhey
2/14/2026

No official code found. Best implementation: JasonQSY/3DOI (47 stars, last commit 2024-02-29). 1 more below.

Abstract

Humans can easily understand a single image as depicting multiple potential objects permitting interaction. We use this skill to plan our interactions with the world and accelerate understanding new objects without engaging in interaction. In this paper, we would like to endow machines with the similar ability, so that intelligent agents can better explore the 3D scene or manipulate objects. Our approach is a transformer-based model that predicts the 3D location, physical properties and affordance of objects. To power this model, we collect a dataset with Internet videos, egocentric videos and indoor images to train and validate our approach. Our model yields strong performance on our data, and generalizes well to robotics data.

View on arXivView PDF

Code Implementations(2)

JasonQSY/3DOICommunity100%

[ICCV 2023] Understanding 3D Object Interaction from a Single Image

474PythonMar 29, 20232 years ago
JasonQSY/3DOI-demoCommunity60%

Gradio demo of our paper "Understanding 3D Object Interaction from a Single Image"

31PythonJun 28, 20233 years ago

Similar Papers

Robo-ABC: Affordance Generalization Beyond Categories via Semantic Correspondence for Robot Manipulation

85% match1 repo1012026

HRP: Human Affordances for Robotic Pre-Training

85% match2024

ManipGPT: Is Affordance Segmentation by Large Vision Models Enough for Articulated Object Manipulation?

84% match2024

Which objects help me to act effectively? Reasoning about physically-grounded affordances

84% match2024

VidBot: Learning Generalizable 3D Actions from In-the-Wild 2D Human Videos for Zero-Shot Robotic Manipulation

84% match1 repo532026

iVideoGPT: Interactive VideoGPTs are Scalable World Models

84% match1 repo1872026

Cite this paper

@article{qian2026understanding,
  title  = {Understanding 3D Object Interaction from a Single Image},
  author = {Shengyi Qian and David F. Fouhey},
  year   = {2026},
  eprint = {2305.09664},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/2305.09664},
  journal = {ICCV 2023 2023}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2305.09664.svg)](https://codeofpaper.com/paper/2305.09664)

Discussion