Spatially varying motion blurred fruit image restoration based on an end-to-end multi-scale conditional generative adversarial networks

Abstract

Abstract During the process of automatic fruits picking by the robot in the orchard, the movements of the robot invariably bring the shakes of the camera, resulting in spatially-varying motion blurs that may impair the accuracy of fruit detection and localization. Therefore, this research proposes a spatially-varying motion blurred fruit image restoration approach based on an end-to-end multi-scale conditional generative adversarial network. Firstly, by adopting a multi-scale residual module, it effectively improves the network's ability to extract the features and reduce the network parameters quantity. Then, it builds a generator network with multi-scale residual blocks as the main body. Additionally, to help the generator in synthesizing the simulated images, the relativistic discriminator structures are employed to evaluate the probability that the real data is more realistic than the simulated data. The low quality orchard fruit image restoration is effectively accomplished by the model, in which the weights are obtained by the GoPro training dataset. The model presented in this paper performs better than other widely-used image recovery algorithms in terms of both qualitative and quantitative indicators, which are demonstrated by the simulated and real experiments. Furthermore, this research can also successfully restore other motion-blurred images in the agricultural field.