Deep Learning-Based Virtual Optical Image Generation and Its Application to Early Crop Mapping

Abstract

This paper investigates the potential of cloud-free virtual optical imagery generated using synthetic-aperture radar (SAR) images and conditional generative adversarial networks (CGANs) for early crop mapping, which requires cloud-free optical imagery at the optimal date for classification. A two-stage CGAN approach, including representation and generation stages, is presented to generate virtual Sentinel-2 spectral bands using all available information from Sentinel-1 SAR and Sentinel-2 optical images. The dual-polarization-based radar vegetation index and all available multi-spectral bands of Sentinel-2 imagery are particularly considered for feature extraction in the representation stage. A crop classification experiment using Sentinel-1 and -2 images in Illinois, USA, demonstrated that the use of all available scattering and spectral features achieved the best prediction performance for all spectral bands, including visible, near-infrared, red-edge, and shortwave infrared bands, compared with the cases that only used dual-polarization backscattering coefficients and partial input spectral bands. Early crop mapping with an image time series, including the virtual Sentinel-2 image, yielded satisfactory classification accuracy comparable to the case of using an actual time-series image set, regardless of the different combinations of spectral bands. Therefore, the generation of virtual optical images using the proposed model can be effectively applied to early crop mapping when the availability of cloud-free optical images is limited.