LiDAR-camera-system-based 3D object detection with proposal selection and grid attention pooling

Abstract

3D object detection is an important module for autonomous driving. A LiDAR camera optical system is suitable for accurate object detection, for it provides both 3D structure and 2D texture features. However, as LiDAR and a camera have different sensor properties, it is challenging to generate effective fusion features. Motivated by this, we propose, to the best of our knowledge, a novel LiDAR–camera based 3D object detection method. First, proposal selection is presented to utilize accurate 2D proposals predicted from RGB images to improve the quality of 3D proposals. It contains a (i) proposal addition and (ii) proposal filter. To increase the recall rate, the proposal addition generates extra 3D proposals via back-projecting 2D proposals on LiDAR depth. The proposal filter removes unrelated 3D proposals by matching 2D proposals with intersection-over-union thresholds. Then, considering the LiDAR mechanism, grid attention pooling is employed to estimate weights of grid points from LiDAR and image features to generate salient pooling features. Comparisons and ablation studies demonstrate that the proposed method achieves better performance and benefits the advanced application of a LiDAR camera system.