Strawberry yield monitoring based on a convolutional neural network using high-resolution aerial orthoimages

Abstract

This article presents the results of studies comparing the quality of work of two modern models of convolutional neural networks YOLOv7 and YOLOv8 used to monitor the yield of strawberries. To do this, we used the transfer method of machine learning models on a set of collected data consisting of four classes of development of generative formations of strawberry. As a result of the study, we obtained a data set that contained images of flowers, ovaries, mature and not mature berries. To ensure the balance of classes in the dataset, the Oversampling method was used, which included the generation of new images by applying various operations, such as resizing the image, normalizing brightness and contrast, converting images by rotating them by a certain angle and reflection, random noise addition, Gaussian blur. To collect data (images) in the field, a DJI Phantom 2 quadrocopter with a DJI Zenmuse Gimbal suspension and a GoPro HD HERO3 camera was used. To assess the quality of the YOLOv7 and YOLOv8 models when recognizing specified classes, well-known metrics were used that estimate the proportion of objects found that are really objects of a given class, such as Precision, Recall and mAP. Analysis of the results showed that the mAP metric for all classes of the YOLOv7 convolutional neural network model was 0,6, and the YOLOv8 model was 0,762. Analysis of the test sample images showed that the average absolute percentage error of image recognition of all classes by the YOLOv7 and YOLOv8 models was 9,2%. The most difficult to recognize was class the ovary of strawberries, the average absolute percentage error of which was 13,2%. In further studies, the use of high-resolution stereo cameras is recommended, which will further improve the accuracy of monitoring potential yields due to the possibility of determining the dimensional parameters of strawberry fruits and constructing 3D models of elevation maps using photogrammetry.