Meta‐Optics Based Parallel Convolutional Processing for Neural Network Accelerator

Abstract

AbstractConvolutional neural networks (CNNs) have shown great performance in computer vision tasks, from image classification to pattern recognition. However, superior performance arises at the expense of high computational costs, which restricts their employment in real‐time decision‐making applications. Computationally intensive convolutions can be offloaded to optical metasurfaces, enabling sub‐picosecond latency and nearly zero energy consumption, but the currently reported approaches require additional bulk optics and can only process polarized light, which limits their practical usages in integrated lightweight systems. To solve these challenges, a novel design of the metasurface‐based optical convolutional accelerator is experimentally demonstrated, offering an ultra‐compact volume of 0.016 , a low cross‐talk of ‐20 dB, polarization insensitivity, and is capable of implementing multiple convolution operations and extracting simultaneously various features from light‐encoded images. The ultra‐compact metasurface‐based optical accelerator can be compactly integrated with a digital imaging system to constitute an optical‐electronic hybrid CNN, which experimentally achieves a consistent accuracy of 96 % in arbitrarily polarized MNIST digits classification. The proposed ultra‐compact metasurface‐based optical convolutional accelerator paves the way for power‐efficient edge‐computing platforms for a range of machine vision applications.