Affiliation:
1. State Key Laboratory of ASIC and System, Fudan University, Shanghai 201203, China
2. College of Electronics and Information Engineering, Tongji University, Shanghai 201804, China
Abstract
Light detection and ranging (LiDAR) technology, a cutting-edge advancement in mobile applications, presents a myriad of compelling use cases, including enhancing low-light photography, capturing and sharing 3D images of fascinating objects, and elevating the overall augmented reality (AR) experience. However, its widespread adoption has been hindered by the prohibitive costs and substantial power consumption associated with its implementation in mobile devices. To surmount these obstacles, this paper proposes a low-power, low-cost, single-photon avalanche detector (SPAD)-based system-on-chip (SoC) which packages the microlens arrays (MLAs) and a lightweight RGB-guided sparse depth imaging completion neural network for 3D LiDAR imaging. The proposed SoC integrates an 8 × 8 SPAD macropixel array with time-to-digital converters (TDCs) and a charge pump, fabricated using a 180 nm bipolar-CMOS-DMOS (BCD) process. Initially, the primary function of this SoC was limited to serving as a ranging sensor. A random MLA-based homogenizing diffuser efficiently transforms Gaussian beams into flat-topped beams with a 45° field of view (FOV), enabling flash projection at the transmitter. To further enhance resolution and broaden application possibilities, a lightweight neural network employing RGB-guided sparse depth complementation is proposed, enabling a substantial expansion of image resolution from 8 × 8 to quarter video graphics array level (QVGA; 256 × 256). Experimental results demonstrate the effectiveness and stability of the hardware encompassing the SoC and optical system, as well as the lightweight features and accuracy of the algorithmic neural network. The state-of-the-art SoC-neural network solution offers a promising and inspiring foundation for developing consumer-level 3D imaging applications on mobile devices.
Subject
Electrical and Electronic Engineering,Biochemistry,Instrumentation,Atomic and Molecular Physics, and Optics,Analytical Chemistry
Reference29 articles.
1. David, R., Allard, B., Branca, X., and Joubert, C. (2020, January 24–26). Study and design of an integrated CMOS laser diode driver for an itof-based 3D image sensor. Proceedings of the CIPS 2020; 11th International Conference on Integrated Power Electronics Systems, Berlin, Germany.
2. A Review of Indirect Time-of-Flight Technologies;Bamji;IEEE Trans. Electron Devices,2022
3. A 240 × 160 3D-stacked SPAD dToF image sensor with rolling shutter and in-pixel histogram for mobile devices;Zhang;IEEE Open J. Solid-State Circuits Soc.,2021
4. A Progress Review on Solid-State LiDAR and Nanophotonics-Based LiDAR Sensors;Li;Laser Photonics Rev.,2022
5. A reconfigurable 3-D-stacked SPAD imager with in-pixel histogramming for flash LIDAR or high-speed time-of-flight imaging;Hutchings;IEEE J. -Solid-State Circuits,2019