ViST: A Ubiquitous Model with Multimodal Fusion for Crop Growth Prediction

Abstract

Crop growth prediction can help agricultural workers to make accurate and reasonable decisions on farming activities. Existing crop growth prediction models focus on one crop and train a single model for each crop. In this article, we develop a ubiquitous growth prediction model for multiple crops, aiming at training a single model for multiple crops. A ubiquitous vision and sensor transformer (ViST) model for crop growth prediction with image and sensor data is developed to achieve the goals. In the proposed model, a cross-attention mechanism is proposed to facilitate the fusion of multimodal feature maps to reduce computational costs and balance the interactive effects among features. To train the model, we combine the data from multiple crops to create a single (ViST) model. A sensor network system is established for data collection on the farm where rice, soybean, and maize are cultivated. Experimental results show that the proposed ViST model has an excellent ubiquitous ability for crop growth prediction with multiple crops.