Affiliation:
1. Princeton University
2. King Abdullah University of Science and Technology, Saudi Arabia
Abstract
Unwanted camera obstruction can severely degrade captured images, including both scene occluders near the camera and partial occlusions of the camera cover glass. Such occlusions can cause catastrophic failures for various scene understanding tasks such as semantic segmentation, object detection, and depth estimation. Existing camera arrays capture multiple redundant views of a scene to see around thin occlusions. Such multi-camera systems effectively form a large synthetic aperture, which can suppress nearby occluders with a large defocus blur, but significantly increase the overall form factor of the imaging setup. In this work, we propose a
monocular single-shot
imaging approach that
optically cloaks
obstructions by emulating a large array. Instead of relying on different camera views, we learn a diffractive optical element (DOE) that performs depth-dependent optical encoding, scattering nearby occlusions while allowing paraxial wavefronts to be focused. We computationally reconstruct unobstructed images from these superposed measurements with a neural network that is trained jointly with the optical layer of the proposed imaging system. We assess the proposed method in simulation and with an experimental prototype, validating that the proposed computational camera is capable of recovering occluded scene information in the presence of severe camera obstruction.
Publisher
Association for Computing Machinery (ACM)
Subject
Computer Graphics and Computer-Aided Design
Reference77 articles.
1. Polka Lines: Learning Structured Illumination and Reconstruction for Active Stereo
2. Single-shot Hyperspectral-Depth Imaging with Learned Diffractive Optics
3. nuScenes: A Multimodal Dataset for Autonomous Driving
4. Wenshan Cai , Uday K Chettiar , Alexander V Kildishev , and Vladimir M Shalaev . 2007. Optical cloaking with metamaterials. Nature photonics 1, 4 ( 2007 ), 224--227. Wenshan Cai, Uday K Chettiar, Alexander V Kildishev, and Vladimir M Shalaev. 2007. Optical cloaking with metamaterials. Nature photonics 1, 4 (2007), 224--227.
5. Ayan Chakrabarti . 2016. Learning sensor multiplexing design through back-propagation. Advances in Neural Information Processing Systems 29 ( 2016 ). Ayan Chakrabarti. 2016. Learning sensor multiplexing design through back-propagation. Advances in Neural Information Processing Systems 29 (2016).
Cited by
8 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献