You Only Need One Thing One Click: Self-Training for Weakly Supervised 3D Scene Understanding

Abstract

Understanding 3D scenes, such as semantic segmentation and instance identification within point clouds, typically demands extensive annotated datasets. However, generating point-by-point labels is an overly laborious process. While recent techniques have been developed to train 3D networks with a minimal fraction of labeled points, our method, dubbed “One Thing One Click,” simplifies this by requiring just a single label per object. To effectively utilize these sparse annotations during network training, we’ve crafted an innovative self-training strategy. This involves alternating between training phases and label spreading, powered by a graph propagation module. Additionally, we integrate a relation network to create category-specific prototypes, improving pseudo label accuracy and steering the training process. Our approach also seamlessly integrates with 3D instance segmentation, incorporating a point-clustering technique. Our method demonstrates superior performance over other weakly supervised strategies for 3D semantic and instance segmentation, as evidenced by tests on both ScanNet-v2 and S3DIS datasets. Remarkably, the efficacy of our self-training method with limited annotations rivals that of fully supervised models. Codes and models are available at https://github.com/liuzhengzhe/One-Thing-One-Click .