SEGMENTATION OF APPLE POINT CLOUDS BASED ON ROI IN RGB IMAGES

Abstract

Autonomous harvesting and evaluation of apples reduce the labour cost. Segmentation of apple point clouds from consumer-grade RGB-D camera is the most important and challenging step in the harvesting process due to the complex structure of apple trees. This paper put forward a segmentation method of apple point clouds based on regions of interest (ROI) in RGB images. Firstly, an annotated RGB dataset of apple trees was built and applied to train the optimized Faster R-CNN to locate ROI containing apples in RGB images. Secondly, the relationship between RGB images and depth images was built to roughly segment the apple point clouds by ROI. Finally, the quality control procedure (QCP) was proposed to improve the quality of segmented apple point clouds. Images for training mainly included two lighting condition, two colours and three apple varieties in orchard, making this method more suitable for practical applications. QCP performed well in filtering noise points and achieved Purity as 96.7% and 96.2% for red and green apples, respectively. Through the comparison method, experimental results indicated that the segmentation method based on ROI is more effective and accurate for red and green apples in orchard. The segmentation method of point clouds based on ROI has great potential for segmentation of point clouds in unstructured scenes.