Semantic-wise hybrid attention generative adversarial network for image inpainting

Abstract

Abstract Deep learning based image inpainting methods can synthesize plausible results based on known information of undamaged areas of the input image. However, most of the existing methods fail to generate high quality structures and textures due to a lack of fully exploring the potential high level semantic information of the generator. To address this issue, we first propose a novel Semantic-Wise Hybrid Attention Generative Adversarial Network (SWHA-GAN), which leverages a lightweight single-stream GAN framework to reconstruct reasonable and natural results. Secondly, a Semantic-Wise Hybrid Attention module is designed to simultaneously promote correct structure and rich details of the generated result. The channel self-attention emphasizes long range semantic dependencies among channels to improve structure accuracy, and spatial attention ensures texture inpainting. Experiments on multiple datasets including faces and natural images show that the proposed SWHA-GAN can generate higher quality results with more details than the state-of-the-arts.