Semantic-guided polarization image fusion method based on a dual-discriminator GAN

Abstract

Polarization image fusion is the process of fusing an intensity image and a polarization parameter image solved by Stokes vector into a more detailed image. Conventional polarization image fusion strategies lack the targeting and robustness for fusing different targets in the images because they do not account for the differences in the characterization of the polarization properties of different materials, and the fusion rule is manually designed. Therefore, we propose a novel end-to-end network model called a semantic guided dual discriminator generative adversarial network (SGPF-GAN) to solve the polarization image fusion problem. We have specifically created a polarization image information quality discriminator (PIQD) block to guide the fusion process by employing this block in a weighted way. The network establishes an adversarial game relationship between a generator and two discriminators. The goal of the generator is to generate a fused image by weighted fusion of each semantic object of the image, the dual discriminator’s objective is to identify specific modalities (polarization/intensity) of various semantic targets. The results of qualitative and quantitative evaluations demonstrate the superiority of our SGPF-GAN in terms of visual effects and quantitative measures. Additionally, using this fusion approach to transparent, camouflaged hidden target detection and image segmentation can significantly boost the performance.