Virtual Inpainting for Dazu Rock Carvings Based on a Sample Dataset

Abstract

Numerous image inpainting algorithms are guided by a basic assumption that the known region in the original image itself can provide sufficient prior information for the guess recovery of the unknown part, which is not often the case in actual art-image inpainting. Sometimes, the art image that needs to be inpainted is so badly damaged that there is little prior information to serve as a good model to infer the appearance of the unknown fragment. Focusing on the lookup strategy for optimal patches, a novel semi-automatic exemplar-based inpainting framework based on a sample dataset is proposed in this article to solve such a problem with three steps: (1) reference images selection from the dataset using deep convolutional network, (2) sample image creation based on reference images with melding algorithm, and (3) exemplar-based inpainting according to the created sample image. Several comparative experiments over Dazu Rock Carvings with the state-of-the-art image completion approaches demonstrate the effectiveness of our contributions. First, the search space for candidate patches is extended from the known region to a sample image. It performs effectively for the inpainting case of little prior information existing in the original image itself. Furthermore, sample image creation is added to reduce the complexity of inpainting via multiple images and avoid the taboo of complete duplication in art restoration. Moreover, Poisson blending is used for post-procedure to improve the visual harmony between the reconstructed fragment and the known region in both color and illumination. Last but not least, our method is successfully applied in the virtual inpainting of Dazu Buddhist face images. The inpainted proposals can be a reference for the final actual artificial inpainting as well as a base for VR show.