Deep Learning-Based Occlusion Handling of Overlapped Plants for Robotic Grasping

Abstract

Instance segmentation of overlapping plants to detect their grasps for possible robotic grasping presents a challenging task due to the need to address the problem of occlusion. We addressed the problem of occlusion using a powerful convolutional neural network for segmenting objects with complex forms and occlusions. The network was trained with a novel dataset named the “occluded plants” dataset, containing real and synthetic images of plant cuttings on flat surfaces with differing degrees of occlusion. The synthetic images were created using the novel framework for synthesizing 2D images by using all plant cutting instances of available real images. In addition to the method for occlusion handling for overlapped plants, we present a novel method for determining the grasps of segmented plant cuttings that is based on conventional image processing. The result of the employed instance segmentation network on our plant dataset shows that it can accurately segment the overlapped plants, and it has a robust performance for different levels of occlusions. The presented plants’ grasp detection method achieved 94% on the rectangle metric which had an angular deviation of 30 degrees and an IoU of 0.50. The achieved results show the viability of our approach on plant species with an irregular shape and provide confidence that the presented method can provide a basis for various applications in the food and agricultural industries.