DeTformer: A Novel Efficient Transformer Framework for Image Deraining

Abstract

Abstract Captured rainy images severely degrade outdoor vision systems performance, such as semi-autonomous or autonomous driving systems and video surveillance systems. Consequently, removing heavy and complex rain streaks i.e., undesirable rainy artifacts from a rainy image plays a crucial role for many high-level computer vision tasks and has drawn researchers’ attention from the past few years. The main drawbacks of Convolutional neural networks: have smaller receptive field, lack the model’s ability to capture long-range dependencies and complicated rainy artifacts, non-adaptive to input content and also computational complexity grows quadratically with input image size. These factors limit the deraining model performance improvement further. Recently, transformer has achieved better performance in both Natural language processing (NLP) and high-level computer vision (CV). We cannot adopt transformer directly to image deraining task as it has following limitations: a) although the transformer possesses powerful long-range computational capability, it lacks the ability to model local features b) to process input image, transformer uses fixed patch size, therefore pixels at the patch edges cannot use local features of surrounding pixels while removing heavy rain streaks. To address these issues, in single image deraining, we proposed a novel and efficient De-raining Transformer (DeTformer). In DeTformer, we designed a “Gated-Depth-wise Convolution Feed-forward Network” (GDWCFN) to address the first issue and applied depthwise convolution to improve the modelling capability of local features and suppress unnecessary features and allow only useful information further. Also, the second issue was addressed, by introducing multi-resolution features in our network, and we applied progressive learning in the transformer and thus it allows the edge pixels to utilize local features effectively. Furthermore, to integrate the extracted multi-scale features and provide feature interaction across channel dimensions, we introduced a “Multi-head Depth-wise Convolution Transposed Attention” (MDWCTA) module. The proposed model experimented with various de-rained datasets and compared with various state-of-the-art models. The experimental results show that DeTformer network achieves superior performance compared to state-of-the-art networks on synthetic and real-world rain datasets.