ARKit LabelMaker: A New Scale for Indoor 3D Scene Understanding

Abstract

Neural network performance scales with both model size and data volume, as shown in both language and image processing. This requires scaling-friendly architectures and large datasets. While transformers have been adapted for 3D vision, a ‘GPT-moment’ remains elusive due to limited training data. We introduce ARKit LabelMaker, a large-scale real-world 3D dataset with dense semantic annotation that is more than three times larger than prior largest dataset. Specifically, we extend ARKitScenes [4] with automatically generated dense 3D labels using an extended LabelMaker pipeline [37], tailored for large-scale pre-training. Training on our dataset improves accuracy across architectures, achieving state-of-the-art 3D semantic segmentation scores on ScanNet and ScanNet200, with notable gains on tail classes. Our code is available at labelmaker.org and our dataset at huggingface.