Deep Neural Networks for Quantitative Damage Evaluation of Building Losses Using Aerial Oblique Images: Case Study on the Great Wall (China)

Abstract

Automated damage evaluation is of great importance in the maintenance and preservation of heritage structures. Damage investigation of large cultural buildings is time-consuming and labor-intensive, meaning that many buildings are not repaired in a timely manner. Additionally, some buildings in harsh environments are impossible to reach, increasing the difficulty of damage investigation. Oblique images facilitate damage detection in large buildings, yet quantitative damage information, such as area or volume, is difficult to generate. In this paper, we propose a method for quantitative damage evaluation of large heritage buildings in wild areas with repetitive structures based on drone images. Unlike existing methods that focus on building surfaces, we study the damage of building components and extract hidden linear symmetry information, which is useful for localizing missing parts in architectural restoration. First, we reconstruct a 3D mesh model based on the photogrammetric method using high-resolution oblique images captured by drone. Second, we extract 3D objects by applying advanced deep learning methods to the images and projecting the 2D object segmentation results to 3D mesh models. For accurate 2D object extraction, we propose an edge-enhanced method to improve the segmentation accuracy of object edges. 3D object fragments from multiple views are integrated to build complete individual objects according to the geometric features. Third, the damage condition of objects is estimated in 3D space by calculating the volume reduction. To obtain the damage condition of an entire building, we define the damage degree in three levels: no or slight damage, moderate damage and severe damage, and then collect statistics on the number of damaged objects at each level. Finally, through an analysis of the building structure, we extract the linear symmetry surface from the remaining damaged objects and use the symmetry surface to localize the positions of missing objects. This procedure was tested and validated in a case study (the Jiankou Great Wall in China). The experimental results show that in terms of segmentation accuracy, our method obtains results of 93.23% mAP and 84.21% mIoU on oblique images and 72.45% mIoU on the 3D mesh model. Moreover, the proposed method shows effectiveness in performing damage assessment of objects and missing part localization.