InNeRF360: Text-Guided 3D-Consistent Object Inpainting on 360° Neural Radiance Fields

Abstract

We propose InNeRF360, an automatic system that accu-rately removes text-specified objectsfrom 360° Neural Radi-ance Fields (NeRF). The challenge is to effectively remove objects while inpainting perceptually consistent content for the missing regions, which is particularly demanding for existing NeRF models due to their implicit volumetric rep-resentation. Moreover, unbounded scenes are more prone to floater artifacts in the inpainted region than frontal-facing scenes, as the change of object appearance and background across views is more sensitive to inaccurate segmentations and inconsistent inpainting. With a trained NeRF and a text description, our method efficiently removes specified ob-jects and inpaints visually consistent content without arti-facts. We apply depth-space warping to enforce consistency across multiview text-encoded segmentations, and then re-fine the inpainted NeRF model using perceptual priors and 3D diffusion-based geometric priors to ensure visual plau-sibility. Through extensive experiments in segmentation and inpainting on 360° and frontal-facing NeRFs, we show that our approach is effective and enhances NeRF's ed-itability. Project page: https://ivr1.github.io/InNeRF360/.