Non-local affinity adaptive acceleration propagation network for generating dense depth maps from LiDAR

Abstract

Depth completion aims to generate dense depth maps from the sparse depth images generated by LiDAR. In this paper, we propose a non-local affinity adaptive accelerated (NL-3A) propagation network for depth completion to solve the mixing depth problem of different objects on the depth boundary. In the network, we design the NL-3A prediction layer to predict the initial dense depth maps and their reliability, non-local neighbors and affinities of each pixel, and learnable normalization factors. Compared with the traditional fixed-neighbor affinity refinement scheme, the non-local neighbors predicted by the network can overcome the propagation error problem of mixed depth objects. Subsequently, we combine the learnable normalized propagation of non-local neighbor affinity with pixel depth reliability in the NL-3A propagation layer, so that it can adaptively adjust the propagation weight of each neighbor during the propagation process, which enhances the robustness of the network. Finally, we design an accelerated propagation model. This model enables parallel propagation of all neighbor affinities and improves the efficiency of refining dense depth maps. Experiments on KITTI depth completion and NYU Depth V2 datasets show that our network is superior to most algorithms in terms of accuracy and efficiency of depth completion. In particular, we predict and reconstruct more smoothly and consistently at the pixel edges of different objects.