Defect Detection in Food Using Multispectral and High-Definition Imaging Combined with a Newly Developed Deep Learning Model

Abstract

The automatic detection of defects (cortical fibers) in pickled mustard tubers (Chinese Zhacai) remains a challenge. Moreover, few papers have discussed detection based on the segmentation of the physical characteristics of this food. In this study, we designate cortical fibers in pickled mustard as the target class, while considering the background and the edible portion of pickled mustard as other classes. We attempt to realize an automatic defect-detection system to accurately and rapidly detect cortical fibers in pickled mustard based on multiple images combined with a UNet4+ segmentation model. A multispectral sensor (MS) covering nine wavebands with a resolution of 870 × 750 pixels and an imaging speed over two frames per second and a high-definition (HD), 4096 × 3000 pixel resolution imaging system were applied to obtain MS and HD images of 200 pickled mustard tuber samples. An improved imaging fusion method was applied to fuse the MS with HD images. After image fusion and other preprocessing methods, each image contained a target; 150 images were randomly selected as the training data and 50 images as the test data. Furthermore, a segmentation model called UNet4+ was developed to detect the cortical fibers in the pickled mustard tubers. Finally, the UNet4+ model was tested on three types of datasets (MS, HD, and fusion images), and the detection results were compared based on Recall, Precision, and Dice values. Our study indicates that the model can successfully detect cortical fibers within about a 30 ± 3 ms timeframe for each type of image. Among the three types of images, the fusion images achieved the highest mean average Dice value of 73.91% for the cortical fibers. At the same time, we compared the UNet4+ model with the UNet++ and UNet3+ models using the same fusion data; the results show that our model achieved better prediction performance for the Dice values, i.e., 9.72% and 27.41% higher than those of the UNet++ and UNet3+ models, respectively.