A Novel Dual U-Net Generative Adversarial Network for Image Inpainting

Abstract

Thanks to the rapid development of deep learning in recent years, image inpainting has made significant progress. As a fundamental task in the field of computer vision, many researchers are committed to exploring more efficient methods, and state-of-the-art research results prove that generative adversarial networks (GAN) have superior performance. However, due to the inherent ill-posedness of image inpainting tasks, these approaches suffer from lack of detailed information, local structural fractures or boundary artifacts. In this paper, we leverage the properties of GAN architecture to process images in more detail and more comprehensively. A novel dual U-Net GAN is designed to inpaint images, which is composed of a U-Net based generator and a U-Net-based discriminator. The former captures semantic information of different scales layer by layer and decodes it back to the original size to repair damaged images, while the latter optimizes the network by combining reconstruction loss, adversarial loss, perceptual loss and style loss. In particular, the U-Net-based discriminator allows per-pixel detail and global feedback to be provided to the generator, guaranteeing the global consistency of the inpainted image and the realism of local shapes and textures. Extensive experiments demonstrate that for different proportions of damage, the images inpainted by our proposed model have reasonable texture structure and contextual semantic information. Furthermore, the proposed model outperforms state-of-the-art models in both qualitative and quantitative comparisons. The code will be available at https://github.com/yjjswu .