SNeRL: Semantic-aware Neural Radiance Fields for Reinforcement Learning

D. Shim, Seungjae Lee, H. J. Kim
2/12/2026

No official code found. Best implementation: jayLEE0301/snerl_official (32 stars, last commit 2023-12-21).

Abstract

As previous representations for reinforcement learning cannot effectively incorporate a human-intuitive understanding of the 3D environment, they usually suffer from sub-optimal performances. In this paper, we present Semantic-aware Neural Radiance Fields for Reinforcement Learning (SNeRL), which jointly optimizes semantic-aware neural radiance fields (NeRF) with a convolutional encoder to learn 3D-aware neural implicit representation from multi-view images. We introduce 3D semantic and distilled feature fields in parallel to the RGB radiance fields in NeRF to learn semantic and object-centric representation for reinforcement learning. SNeRL outperforms not only previous pixel-based representations but also recent 3D-aware representations both in model-free and model-based reinforcement learning.

View on arXivView PDF

Code Implementations(1)

jayLEE0301/snerl_officialCommunity65%

Official code for "SNeRL: Semantic-aware Neural Radiance Fields for Reinforcement Learning" (ICML 2023)

326Shell, PythonMay 20, 20232 years ago
neural-radiance-fieldsreinforcement-learningrepresentation-learning

Similar Papers

Semantic Ray: Learning a Generalizable Semantic Field with Cross-Reprojection Attention

86% match2026

Content-Aware Radiance Fields: Aligning Model Complexity with Scene Intricacy Through Learned Bitwidth Quantization

85% match1 repo32026

CtrlNeRF: The Generative Neural Radiation Fields for the Controllable Synthesis of High-fidelity 3D-Aware Images

84% match1 repo12024

Learning Neural Duplex Radiance Fields for Real-Time View Synthesis

83% match2026

GenNBV: Generalizable Next-Best-View Policy for Active 3D Reconstruction

83% match1 repo1102026

Plen-VDB: Memory Efficient VDB-Based Radiance Fields for Fast Training and Rendering

83% match2026

Cite this paper

@article{shim2026snerl,
  title  = {SNeRL: Semantic-aware Neural Radiance Fields for Reinforcement Learning},
  author = {D. Shim and Seungjae Lee and H. J. Kim},
  year   = {2026},
  eprint = {2301.11520},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/pdf/2301.11520},
  journal = {ICML 2023 2023}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2301.11520.svg)](https://codeofpaper.com/paper/2301.11520)

Discussion