A Novel Adversarial Based Hyperspectral and Multispectral Image Fusion

Abstract

In order to reconstruct a high spatial and high spectral resolution image (H2SI), one of the most common methods is to fuse a hyperspectral image (HSI) with a corresponding multispectral image (MSI). To effectively obtain both the spectral correlation of bands in HSI and the spatial correlation of pixels in MSI, this paper proposes an adversarial selection fusion (ASF) method for the HSI–MSI fusion problem. Firstly, the unmixing based fusion (UF) method is adopted to dig out the spatial correlation in MSI. Then, to acquire the spectral correlation in HSI, a band reconstruction-based fusion (BRF) method is proposed, regarding H2SI as the product of the optimized band image dictionary and reconstruction coefficients. Finally, spectral spatial quality (SSQ) index is designed to guide the adversarial selection process of UF and BRF. Experimental results on four real-world images demonstrate that the proposed strategy achieves smaller errors and better reconstruction results than other comparison methods.