Attention‐optimized 3D segmentation and reconstruction system for sewer pipelines employing multi‐view images

Abstract

AbstractExisting deep learning‐based defect inspection results on images lack depth information to fully demonstrate the sewer, despite their high accuracy. To address this limitation, a novel attention‐optimized three‐dimensional (3D) segmentation and reconstruction system for sewer pipelines is presented. First, a real‐time sewer segmentation method called AM‐Pipe‐SegNet is developed to inspect defects (i.e., misalignment, obstacle, and fracture) efficiently. Attention mechanisms (AMs) are introduced to improve the performance of segmentation. Second, an attention‐optimized and sparse‐initialized depth estimation network called AM‐Pipe‐DepNet is presented to generate depth maps from multi‐view images. Third, a 2D‐to‐3D mapping algorithm is proposed to remove noise and transform the sewer segmentation results into 3D spaces. Comparison experiments reveal that incorporating AMs into the network significantly enhances pipe segmentation and 3D reconstruction performance. Finally, two digital replicas of real sewer pipes are built based on photos taken by probes, providing valuable insights for sewer maintenance.