Single‐image snow removal algorithm based on generative adversarial networks

Abstract

AbstractThe effect of snowfall on an image is not only the interference of snow particles but also snow streaks and masking effects (similar to haze). Snowy weather severely reduces the accuracy of computer vision systems. There is a lot of interest in how to effectively remove snow while preserving as much of the original image information as possible. Based on this, the authors propose an effective Generative Adversarial Network (GAN) snow removal algorithm for single images to solve the snow removal failure problem caused by irregular snow particles and snow streaks. Specifically, the authors improve the original GAN network as follows: A novel Transformer module, the Contextual Transformer (CoT) module, is adopted in the residual modules based generator. It effectively uses the contextual information of the snow streaks neighbourhood to restore the texture and information in the noisy image as much as possible based on the focus on snow streaks features. Also, using learnable Regionalized Normalization (RN‐L), potentially corrupted and undamaged regions are automatically detected for separate normalization, and global affine transformations are performed to enhance their fusion. In addition, a multi‐scale discriminator is used in the discriminator to make the discrimination more adequate and retain more details. Extensive experiments have shown that the authors’ GAN network snow removal algorithm outperforms various current networks on snow removal studies in terms of evaluation metrics on both synthetic and real datasets.