Image quality enhancement of 4D light field microscopy via reference impge propagation-based one-shot learning

Abstract

AbstractFour-dimensional (4D) light-field (LF) microscopes can acquire 3D information about target objects using a microlens array (MLA). However, the resolution and quality of sub-images in the LF images are reduced because of the spatial multiplexing of rays by the element lenses of the MLA. To overcome these limitations, this study proposes an LF one-shot learning technique that can convert LF sub-images into high-quality images similar to the 2D images of conventional optical microscopes obtained without any external training datasets for image enhancement. The proposed convolutional neural network model was trained using only one training dataset comprising a high-resolution reference image captured without an MLA as the ground truth. Further, its input was the central view of the LF image. After LF one-shot learning, the trained model should be able to convert well the other LF sub-images of various directional views that were not used in the main training process. Therefore, novel learning techniques were designed for LF one-shot learning. These novel techniques include an autoencoder-based model initialization method, a feature map-based learning algorithm to prevent the overfitting of the model, and cut loss to prevent saturation. The experimental results verified that the proposed technique effectively enhances the LF image quality and resolution using a reference image. Moreover, this method enhances the resolution by up to 13 times, decreases the noise amplification effect, and restores the lost details of microscopic objects. The proposed technique is stable and yields superior experimental results compared with those of the existing resolution-enhancing methods.