Dual-branch fusion model for lensless imaging

Abstract

A lensless camera is an imaging system that replaces the lens with a mask to reduce thickness, weight, and cost compared to a lensed camera. The improvement of image reconstruction is an important topic in lensless imaging. Model-based approach and pure data-driven deep neural network (DNN) are regarded as two mainstream reconstruction schemes. In this paper, the advantages and disadvantages of these two methods are investigated to propose a parallel dual-branch fusion model. The model-based method and the data-driven method serve as two independent input branches, and the fusion model is used to extract features from the two branches and merge them for better reconstruction. Two types of fusion model named Merger-Fusion-Model and Separate-Fusion-Model are designed for different scenarios, where Separate-Fusion-Model is able to adaptively allocate the weights of the two branches by the attention module. Additionally, we introduce a novel network architecture named UNet-FC into the data-driven branch, which enhances reconstruction by making full use of the multiplexing property of lensless optics. The superiority of the dual-branch fusion model is verified by drawing comparison with other state-of-the-art methods on public dataset (+2.95dB peak signal-to-noise (PSNR), +0.036 structural similarity index (SSIM), −0.0172 Learned Perceptual Image Patch Similarity (LPIPS)). Finally, a lensless camera prototype is constructed to further validate the effectiveness of our method in a real lensless imaging system.