A Spatial–Temporal Bayesian Deep Image Prior Model for Moderate Resolution Imaging Spectroradiometer Temporal Mixture Analysis

Abstract

Time-series remote sensing images are important in agricultural monitoring and investigation. However, most time-series data with high temporal resolution have the problem of insufficient spatial resolution which cannot meet the requirement of precision agriculture. The unmixing technique can obtain the object abundances with richer spatial information from the coarse-resolution images. Although the unmixing technique is widely used in hyperspectral data, it is insufficiently researched for time-series data. Temporal unmixing extends spectral unmixing to the time domain from the spectral domain, and describes the temporal characteristics rather than the spectral characteristics of different ground objects. Deep learning (DL) techniques have achieved promising performance for the unmixing problem in recent years, but there are still few studies on temporal mixture analysis (TMA), especially in the application of crop phenological monitoring. This paper presents a novel spatial–temporal deep image prior method based on a Bayesian framework (ST-Bdip), which innovatively combines the knowledge-driven TMA model and the DL-driven model. The normalized difference vegetation index (NDVI) time series of moderate resolution imaging spectroradiometer (MODIS) data is used as the object for TMA, while the extracted seasonal crop signatures and the fractional coverages are perceived as the temporal endmembers (tEMs) and corresponding abundances. The proposed ST-Bdip method mainly includes the following contributions. First, a deep image prior model based on U-Net architecture is designed to efficiently learn the spatial context information, which enhances the representation of abundance modeling compared to the traditional non-negative least squares algorithm. Second, The TMA model is incorporated into the U-Net training process to exploit the knowledge in the forward temporal model effectively. Third, the temporal noise heterogeneity in time-series images is considered in the model optimization process. Specifically, the anisotropic covariance matrix of observations from different time dimensions is modeled as a multivariate Gaussian distribution and incorporated into the calculation of the loss function. Fourth, the "purified means" approach is used to further optimize crop tEMs and the corresponding abundances. Finally, the expectation–maximization (EM) algorithm is designed to solve the maximum a posterior (MAP) problem of the model in the Bayesian framework. Experimental results on three synthetic datasets with different noise levels and two real MODIS datasets demonstrate the superiority of the proposed approach in comparison with seven traditional and advanced unmixing algorithms.