Light field camera calibration and point spread function calculation based on differentiable ray tracing

Abstract

The imaging process of the light field (LF) camera with a micro-lens array (MLA) may suffer from multiple aberrations. It is thus difficult to precisely calibrate the intrinsic hardware parameters and calculate the corresponding point spread function (PSF). To build an aberration-aware solution with better generalization, we propose an end-to-end imaging model based on the differentiable ray tracing. The input end is the point source location, and the output end is the rendered LF image, namely, PSF. Specially, a projection method is incorporated into the imaging model, eliminating the huge memory overhead induced by a large array of periodic elements. Taking captured PSF images as the ground truth, the LF camera is calibrated with the genetic algorithm initially and then the gradient-based optimization. This method is promising to be used in various LF camera applications, especially in challenging imaging conditions with severe aberrations.