Investigating deep optics model representation in affecting resolved all-in-focus image quality and depth estimation fidelity

Abstract

The end-to-end (E2E) optimization of optics and image processing, dubbed deep optics, has renewed the state-of-the-art in various computer vision tasks. However, specifying the proper model representation or parameterization of the optical elements remains elusive. This article comprehensibly investigates three modeling hypotheses of the phase coded-aperture imaging under a representative context of deep optics, joint all-in-focus (AiF) imaging and monocular depth estimation (MDE). Specifically, we analyze the respective trade-off of these models and provide insights into relevant domain-specific requirements, explore the connection between the spatial feature of the point spread function (PSF) and the performance trade-off between the AiF and MDE tasks, and discuss the model sensitivity to possible fabrication errors. This study provides new prospects for future deep optics designs, particularly those aiming for AiF and/or MDE.