Sar2color: Learning Imaging Characteristics of SAR Images for SAR-to-Optical Transformation

Abstract

Optical images are rich in spectral information, but difficult to acquire under all-weather conditions, while SAR images can overcome adverse meteorological conditions, but geometric distortion and speckle noise will reduce the quality of SAR images and thus make image interpretation more challenging. Therefore, transforming SAR images to optical images to assist SAR image interpretation will bring opportunities for SAR image application. With the advancement of deep learning technology, the ability of SAR-to-optical transformation has been greatly improved. However, most of the current mainstream transformation methods do not consider the imaging characteristics of SAR images, and there will be failures such as noisy color spots and regional landform deformation in the generated optical images. Moreover, since the SAR image itself does not contain color information, there also exist many color errors in these results. Aiming at the above problems, Sar2color, an end-to-end general SAR-to-optical transformation model, is proposed based on a conditional generative adversarial network (CGAN). The model uses DCT residual block to reduce the effect of coherent speckle noise on the generated optical images, and constructs the Light atrous spatial pyramid pooling (Light-ASPP) module to mitigate the negative effect of geometric distortion on the generation of optical images. These two designs ensure the precision of texture details when the SAR image is transformed into an optical image, and use the correct color memory block (CCMB) to improve the color accuracy of transformation results. Towards the Sar2color model, we have carried out evaluations on the homologous heterogeneous SAR image and optical image pairing dataset SEN1-2. The experimental results show that, compared with other mainstream transformation models, Sar2color achieves the state-of-the-art effect on all three objective and one subjective evaluation metrics. Furthermore, we have carried out various ablation experiments, and the results show the effectiveness of each designed module of Sar2color.