Method of depth simulation imaging and depth image super-resolution reconstruction for a 2D/3D compatible CMOS image sensor

Abstract

This paper presents a depth simulation imaging and depth image super-resolution (SR) method for two-dimensional/three-dimensional compatible CMOS image sensors. A depth perception model is established to analyze the effects of depth imaging parameters and evaluate the real imaging effects. We verify its validity by analyzing the depth error, imaging simulation, and auxiliary physical verification. By means of the depth simulation images, we then propose a depth SR reconstruction algorithm to recover the low-resolution depth maps to the high-resolution depth maps in two types of datasets. With the best situation in depth accuracy kept, the root mean square error (RMSE) of Middlebury dataset images are 0.0156, 0.0179, and 0.0183 m. The RMSE of RGB-D dataset images are 0.0223 and 0.0229 m. Compared with other listed conventional algorithms, our algorithm reduces the RMSE by more than 16.35%, 17.19%, and 23.90% in the Middlebury dataset images. Besides, our algorithm reduces the RMSE by more than 9.71% and 8.76% in the RGB-D dataset images. The recovery effects achieve optimized results.