Semantically-aware Neural Radiance Fields for Visual Scene Understanding: A Comprehensive Review

Thang-Anh-Quan Nguyen, Amine Bourki, Mátyás Macudzinski, Anthony Brunel, Mohammed Bennamoun
2/17/2024
cs.CV

No official code found. Best implementation: abourki/SoTA-Semantically-aware-NeRFs (15 stars, last commit 2025-10-13).

Abstract

This review thoroughly examines the role of semantically-aware Neural Radiance Fields (NeRFs) in visual scene understanding, covering an analysis of over 250 scholarly papers. It explores how NeRFs adeptly infer 3D representations for both stationary and dynamic objects in a scene. This capability is pivotal for generating high-quality new viewpoints, completing missing scene details (inpainting), conducting comprehensive scene segmentation (panoptic segmentation), predicting 3D bounding boxes, editing 3D scenes, and extracting object-centric 3D models. A significant aspect of this study is the application of semantic labels as viewpoint-invariant functions, which effectively map spatial coordinates to a spectrum of semantic labels, thus facilitating the recognition of distinct objects within the scene. Overall, this survey highlights the progression and diverse applications of semantically-aware neural radiance fields in the context of visual scene interpretation.

View on arXivView PDF

Code Implementations(1)

abourki/SoTA-Semantically-aware-NeRFsCommunity100%

🚀🚀🚀 [Journal Pre-print] Semantically-aware Neural Radiance Fields for Visual Scene Understanding: A Comprehensive Review

150Feb 13, 20241 years ago

Similar Papers

GP-NeRF: Generalized Perception NeRF for Context-Aware 3D Scene Understanding

85% match1 repo282026

Semantic Ray: Learning a Generalizable Semantic Field with Cross-Reprojection Attention

85% match2026

In-Place Panoptic Radiance Field Segmentation with Perceptual Prior for 3D Scene Understanding

84% match2024

RT-GS2: Real-Time Generalizable Semantic Segmentation for 3D Gaussian Representations of Radiance Fields

84% match1 repo252024

Semantic Gaussians: Open-Vocabulary Scene Understanding with 3D Gaussian Splatting

84% match1 repo2742024

Lift3D: Zero-Shot Lifting of Any 2D Vision Model to 3D

83% match2026

Cite this paper

@article{nguyen2024semanticallyaware,
  title  = {Semantically-aware Neural Radiance Fields for Visual Scene Understanding: A Comprehensive Review},
  author = {Thang-Anh-Quan Nguyen and Amine Bourki and Mátyás Macudzinski and Anthony Brunel and Mohammed Bennamoun},
  year   = {2024},
  eprint = {2402.11141},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2402.11141v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2402.11141.svg)](https://codeofpaper.com/paper/2402.11141)

Discussion