Segmentation Can Aid Detection: Segmentation-Guided Single Stage Detection for 3D Point Cloud

Abstract

Detecting accurate 3D bounding boxes from point cloud data plays an essential role in autonomous driving. However, improving performance requires more complex models, which come with high memory and computational cost. In this work, we design a Segmentation-Guided Auxiliary Network (SGAN) to improve the localization quality of detection. The points from different levels are concatenated to generate the multi-scale feature for the points used for prediction, i.e., candidate points. SGAN is jointly optimized by two tasks of candidate points—segmentation and center estimation—and it is only used in training and therefore introduces no extra computation in the inference stage. Furthermore, we consider that point-based detectors suffer from the outline points of sampling, so we explore the correlation between the data and propose the Point Cloud External Attention (PCEA) to extract the semantic features with a low memory cost. Our method SGSSD achieves a large margin against the baseline on the KITTI and Waymo datasets, and it runs at 25 FPS for inference on the KITTI test set with a single NVIDIA RTX 3090.