3D reconstruction of light-field images based on spatiotemporal correlation super-resolution

Abstract

In this paper, we make full advantage of the information correlation of subaperture images and propose a new super-resolution (SR) reconstruction method based on spatiotemporal correlation to achieve SR reconstruction for light-field images. Meanwhile, the offset compensation method based on optical flow and spatial transformer network is designed to realize accurate compensation between adjacent light-field subaperture images. After that, the obtained light-field images with high resolution are combined with the self-designed system based on phase similarity and SR reconstruction to realize accurate 3D reconstruction of a structured light field. Finally, experimental results demonstrate the validity of the proposed method to perform accurate 3D reconstruction of light-field images from the SR data. Generally, our method makes full use of the redundant information between different subaperture images, hides the upsampling process in the convolution, provides more sufficient information, and reduces time-consuming procedures, which is more efficient to realize the accurate 3D reconstruction of light-field images.