Large-region image inpainting using adaptive Fourier Neural Network and space-frequency domain structural analysis

Abstract

Due to its remarkable performance, the convolutional neural network (CNN) has gained widespread usage in image inpainting challenges. However, most of these CNN-based methods reconstruct images only in the spatial domain, which produces satisfactory outcomes for small-region inpainting tasks, but blurs the details and generates incomplete structures for large-region inpainting tasks with complex backgrounds. In this paper, we address the issue of large-region inpainting tasks by our novel Adaptive Fourier Neural Network. Specifically, in our network, a Fourier-based global receptive field module is introduced to incorporate frequency information and expand the receptive field by transforming local convolutions into global convolutions, enabling the proposed network to transmit global information to the missing region. Furthermore, to better fuse spatial and frequency features, an attention-based joint space-frequency module is proposed to combine spatial and frequency information. Finally, to validate the effectiveness and robustness of our proposed method, we conduct qualitative and quantitative experiments on two popular datasets Paris StreetView and Places. The experimental results demonstrate that our proposed method outperforms state-of-the-art methods by generating sharper, more coherent, and visually plausible inpainting results. Code will be released after this work published: https://github.com/langka9/AFNN.git.