Enhancing single-pixel imaging reconstruction using hybrid transformer network with adaptive feature refinement

Abstract

Single-pixel imaging (SPI) is a novel imaging technique that applies to acquiring spatial information under low light, high absorption, and backscattering conditions. The existing reconstruction techniques, such as pattern analysis and signal-recovery algorithms, are inefficient due to their iterative behaviors and substantial computational requirements. In this paper, we address these issues by proposing a hybrid convolutional-transformer network for efficient and accurate SPI reconstruction. The proposed model has a universal pre-reconstruction layer that can reconstruct the single-pixel measurements collected using any SPI method. Moreover, we utilize the hierarchical encoder-decoder network in U-Net architectures and employ the proposed CONText AggregatIon NEtwoRk (Container) as the adaptive feature refinement module to adaptively leverage the significance of globally and locally enhanced features in SPI reconstruction. As such, we can improve the conventional SPI methods in terms of reconstruction speed and accuracy. Extensive experiments show that the proposed model achieve a significant performance improvement as compared to traditional SPI methods digitally and experimentally while increasing the reconstruction frame rates by threefold. Moreover, the proposed model also outperforms state-of-the-art deep learning models in performing single-pixel imaging reconstruction.