Multi-Level Neural Scene Graphs for Dynamic Urban Environments

Abstract

We estimate the radiance field of large-scale dynamic ar-eas from multiple vehicle captures under varying environ-mental conditions. Previous works in this domain are ei-ther restricted to static environments, do not scale to more than a single short video, or struggle to separately repre-sent dynamic object instances. To this end, we present a novel, decomposable radiance field approach for dynamic urban environments. We propose a multi-level neural scene graph representation that scales to thousands of images from dozens of sequences with hundreds of fast-moving ob-jects. To enable efficient training and rendering of our rep-resentation, we develop a fast composite ray sampling and rendering scheme. To test our approach in urban driving scenarios, we introduce a new, novel view synthesis bench-mark. We show that our approach outperforms prior art by a significant margin on both established and our proposed benchmark while being faster in training and rendering.