SE-COTR: A Novel Fruit Segmentation Model for Green Apples Application in Complex Orchard

Abstract

Because of the unstructured characteristics of natural orchards, the efficient detection and segmentation applications of green fruits remain an essential challenge for intelligent agriculture. Therefore, an innovative fruit segmentation method based on deep learning, termed SE-COTR (segmentation based on coordinate transformer), is proposed to achieve accurate and real-time segmentation of green apples. The lightweight network MobileNetV2 is used as the backbone, combined with the constructed coordinate attention-based coordinate transformer module to enhance the focus on effective features. In addition, joint pyramid upsampling module is optimized for integrating multiscale features, making the model suitable for the detection and segmentation of target fruits with different sizes. Finally, in combination with the outputs of the function heads, the dynamic convolution operation is applied to predict the instance mask. In complex orchard environment with variable conditions, SE-COTR achieves a mean average precision of 61.6% with low complexity for green apple fruit segmentation at severe occlusion and different fruit scales. Especially, the segmentation accuracy for small target fruits reaches 43.3%, which is obviously better than other advanced segmentation models and realizes good recognition results. The proposed method effectively solves the problem of low accuracy and overly complex fruit segmentation models with the same color as the background and can be built in portable mobile devices to undertake accurate and efficient agricultural works in complex orchard.