GAMNet: Global attention via multi‐scale context for depth estimation algorithm and application

Abstract

AbstractDeep neural networks significantly enhance the accuracy of the stereo‐based disparity estimation. Some current methods suffer from inefficient use of the global context information, which will lead to the loss of structural details in ill‐posed areas. To this end, a novel stereo network GAMNet is designed, composed of three core components (GDA, MPF, DCA) for estimating the depth prediction in challenging real‐world environments. First, a lightweight attention module is presented, integrating the global semantic cues for every feature position across the channel and spatial dimensions. Next, the MPF module is constructed to fuse the diverse semantic and contextual information from different levels of the feature pyramid. Finally, cost volume is aggregated with a stacked encoder‐decoder composed of the DCA module and 3D convolutions, filtering the transmission of matching clues and capturing the rich global contexts. Substantial experiments conducted on KITTI 2012, KITTI 2015, SceneFlow, and Middlebury‐v3 datasets manifest that GAMNet surpasses preceding methods with contour‐preserving disparity predictions. In addition, the first 3D scene reconstruction linear evaluation strategy on spatial grasping points for the end‐to‐end stereo networks in an unsupervised mode is proposed, and it is deployed on the designed robot vision‐guided system. In application experiments, the method can produce densely high‐precision 3D reconstructions to implement the grasping task in complex real‐world scenes, and achieve excellent robust performance with competitive inference efficiency.