Single-frame 3D lensless microscopic imaging via deep learning

Abstract

Since the pollen of different species varies in shape and size, visualizing the 3-dimensional structure of a pollen grain can aid in its characterization. Lensless sensing is useful for reducing both optics footprint and cost, while the capability to image pollen grains in 3-dimensions using such a technique could be truly disruptive in the palynology, bioaerosol sensing, and ecology sectors. Here, we show the ability to employ deep learning to generate 3-dimensional images of pollen grains using a series of 2-dimensional images created from 2-dimensional scattering patterns. Using a microscope to obtain 3D Z-stack images of a pollen grain and a 520 nm laser to obtain scattering patterns from the pollen, a single scattering pattern per 3D image was obtained for each position of the pollen grain within the laser beam. In order to create a neural network to transform a single scattering pattern into different 2D images from the Z-stack, additional Z-axis information is required to be added to the scattering pattern. Information was therefore encoded into the scattering pattern image channels, such that the scattering pattern occupied the red channel, and a value indicating the position in the Z-axis occupied the green and blue channels. Following neural network training, 3D images were formed from collated generated 2D images. The volumes of the pollen grains were generated with a mean accuracy of ∼84%. The development of airborne-pollen sensors based on this technique could enable the collection of rich data that would be invaluable to scientists for understanding mechanisms of pollen production climate change and effects on the wider public health.