Weakly Supervised Perennial Weed Detection in a Barley Field

Abstract

In the field of precision agriculture, weed detection models combined with selective spraying by ground or aerial robotics are seen as promising approaches for increasing yield harvests while simultaneously minimizing herbicide use. The available labeled training data are a major bottleneck for developing and applying supervised deep learning models, and more automated methods for labeled data generation are therefore needed. Our research aims to address this need by introducing a rule-based method for label data generation for perennial weeds. For this research, a dataset of a barley field was collected using an unmanned aerial vehicle (UAV) with a flight altitude of 10 m. A multispectral and a thermal camera were used for the data collection. The preprocessed dataset consists of multispectral and thermal orthomosaic images along with a canopy height model. The methodological part of this article introduces a proposed rule-based method for labeled data generation for perennial weeds based on the Normalized Difference Vegetation Index (NDVI), and this approach is further used to generate labels for the measured data. The preprocessed data combined with the generated labels was used to train U-net models. Three data combinations are used for the training and testing: multispectral, multispectral–thermal and multispectral–thermal–canopy–height–model. This approach was used to evaluate whether additional data improve model performance. To evaluate the models on ground truth labels, they are tested on a manually annotated test dataset, which consists of 10% of the whole dataset. The tested models achieved an F1 score of 0.82–0.83 on the test dataset. This finding demonstrates that the rule-based labeling method generates valid labels for the perennial weed detection task. Furthermore, our study shows that data fusion improved the model slightly. The data combination of multispectral–thermal–canopy–height–model as input resulted in the best-performing model, with an F1 score of 0.835.