A deep learning approach for deriving wheat phenology from near-surface RGB image series using spatiotemporal fusion

Abstract

Real-time image series of wheat from near-surface cameras provide high-quality data for monitoring wheat phenological stages. In this study, three deep learning models based on three different spatiotemporal feature fusion methods, namely sequential fusion, synchronous fusion, and parallel fusion, were constructed and evaluated for deriving wheat phenological stages with these near-surface RGB image series. Moreover, the impact of different image resolutions, capture perspectives, and model training strategies on the performance of deep learning models was also investigated. The results indicate that the model using the sequential fusion method is optimal, with an overall accuracy (OA) of 0.935, a mean absolute error (MAE) of 0.069, F1-score (F1) of 0.936, and kappa coefficients (Kappa) of 0.924 in wheat phenological stages. Besides, enhanced image resolution and the suitable image capture perspective introduce more effective features for phenological stage detection, thereby enhancing the model’s accuracy. Furthermore, with regard to the model training, applying a two-step fine-tuning strategy will also enhance the model’s robustness to random variations in perspective.