Progressive Dunhuang murals inpainting based on recurrent feature reasoning network

Abstract

The Dunhuang murals, notably the paintings on the interior walls of China’s Dunhuang Grottoes, are considered international cultural treasure. The Dunhuang murals were ruined to varied degrees after a lengthy period of erosion. Deep learning networks were utilized to reconstruct broken parts of murals in order to better preserve their important historical and cultural values. Due to the presence of various damages, such as large peeling, mold and scratches, and multi-scale objects in the mural, a simple porting of existing working methods is suboptimal. In this paper, we propose a progressive Dunhuang murals inpainting (PDMI) based on recurrent feature reasoning network to progressively infer the pixel values of hole centers by a progressive approach, aiming to obtain visually reasonable and semantically consistent inpainted results. PDMI consists mainly of the FFC-based recurrent feature reasoning (RFR) module and Multi-scale Knowledge Consistent Attention (MKCA) module. The RFR module first fills in the feature value at the feature map’s hole border, then utilizes the obtained feature value as a clue for further inference. The module steadily improved the limitation of hole centers, making the inpainted results more explicit; MKCA enables feature maps in RFR to handle richer background information from distant location information in a flexible manner while preventing misuse. After several round-robin inferences provide multiple feature maps, these feature maps are fused using an adaptive feature weighted fusion mechanism, then the fused feature maps decode back to RGB image. Experiments on a publicly available dataset and a self-made Dunhuang mural dataset reveal that the proposed method outperforms the comparison algorithm in both qualitative and quantitative aspects.