3D scene generation from scene graphs and self-attention

Pietro Bonazzi, Mengqi Wang, Diego Martin Arroyo, Fabian Manhardt, Nico Messikomer +2 more
4/2/2024
cs.CV

No official code found. Best implementation: pbonazzi/graph-scene-transformer (2 stars, last commit 2024-05-16).

Abstract

Synthesizing realistic and diverse indoor 3D scene layouts in a controllable fashion opens up applications in simulated navigation and virtual reality. As concise and robust representations of a scene, scene graphs have proven to be well-suited as the semantic control on the generated layout. We present a variant of the conditional variational autoencoder (cVAE) model to synthesize 3D scenes from scene graphs and floor plans. We exploit the properties of self-attention layers to capture high-level relationships between objects in a scene, and use these as the building blocks of our model. Our model, leverages graph transformers to estimate the size, dimension and orientation of the objects in a room while satisfying relationships in the given scene graph. Our experiments shows self-attention layers leads to sparser (7.9x compared to Graphto3D) and more diverse scenes (16%).

View on arXivView PDF

Code Implementations(1)

pbonazzi/graph-scene-transformerCommunity100%

[Master Project] 3D Scene Generation from Scene Graphs and Graph Self-Attention

20Shell, PythonMay 16, 20242 years ago

Similar Papers

Scene Graph Disentanglement and Composition for Generalizable Complex Image Generation

85% match2 repos72026

Planner3D: LLM-enhanced graph prior meets 3D indoor scene explicit regularization

84% match1 repo62024

GeoSceneGraph: Geometric Scene Graph Diffusion Model for Text-guided 3D Indoor Scene Synthesis

84% match2025

EchoScene: Indoor Scene Generation via Information Echo over Scene Graph Diffusion

83% match1 repo1042026

HiGS: Hierarchical Generative Scene Framework for Multi-Step Associative Semantic Spatial Composition

82% match2025

FreeScene: Mixed Graph Diffusion for 3D Scene Synthesis from Free Prompts

82% match2026

Cite this paper

@article{bonazzi2024scene,
  title  = {3D scene generation from scene graphs and self-attention},
  author = {Pietro Bonazzi and Mengqi Wang and Diego Martin Arroyo and Fabian Manhardt and Nico Messikomer and Federico Tombari and Davide Scaramuzza},
  year   = {2024},
  eprint = {2404.01887},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2404.01887v3}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2404.01887.svg)](https://codeofpaper.com/paper/2404.01887)

Discussion