Neuromorphic-P2M: processing-in-pixel-in-memory paradigm for neuromorphic image sensors

Author:

Kaiser Md Abdullah-Al,Datta Gourav,Wang Zixu,Jacob Ajey P.,Beerel Peter A.,Jaiswal Akhilesh R.

Abstract

Edge devices equipped with computer vision must deal with vast amounts of sensory data with limited computing resources. Hence, researchers have been exploring different energy-efficient solutions such as near-sensor, in-sensor, and in-pixel processing, bringing the computation closer to the sensor. In particular, in-pixel processing embeds the computation capabilities inside the pixel array and achieves high energy efficiency by generating low-level features instead of the raw data stream from CMOS image sensors. Many different in-pixel processing techniques and approaches have been demonstrated on conventional frame-based CMOS imagers; however, the processing-in-pixel approach for neuromorphic vision sensors has not been explored so far. In this work, for the first time, we propose an asynchronous non-von-Neumann analog processing-in-pixel paradigm to perform convolution operations by integrating in-situ multi-bit multi-channel convolution inside the pixel array performing analog multiply and accumulate (MAC) operations that consume significantly less energy than their digital MAC alternative. To make this approach viable, we incorporate the circuit's non-ideality, leakage, and process variations into a novel hardware-algorithm co-design framework that leverages extensive HSpice simulations of our proposed circuit using the GF22nm FD-SOI technology node. We verified our framework on state-of-the-art neuromorphic vision sensor datasets and show that our solution consumes ~2× lower backend-processor energy while maintaining almost similar front-end (sensor) energy on the IBM DVS128-Gesture dataset than the state-of-the-art while maintaining a high test accuracy of 88.36%.

Publisher

Frontiers Media SA

Subject

Computer Science Applications,Biomedical Engineering,Neuroscience (miscellaneous)

Reference54 articles.

1. “A low power, fully event-based gesture recognition system,”;Amir;2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR),2017

2. “Towards autonomous driving: a multi-modal 360° perception proposal,”;Beltrán,2020

3. A burst-mode word-serial address-event link-I: Transmitter design;Boahen;IEEE Trans. Circ. Syst. I Regular Pap.,2004

4. “Fully embedding fast convolutional networks on pixel processor arrays,”;Bose,2020

5. In-sensor computing for machine vision;Chai;Nature,2020

Cited by 3 articles. 订阅此论文施引文献 订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献

1. Hardware-Algorithm Co-Design Enabling Processing-In-Pixel-In-Memory (P2M) for Neuromorphic Vision Sensors;ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP);2024-04-14

2. Design Considerations for 3D Heterogeneous Integration Driven Analog Processing-in-Pixel for Extreme-Edge Intelligence;2023 IEEE International Conference on Rebooting Computing (ICRC);2023-12-05

3. SpikingJelly: An open-source machine learning infrastructure platform for spike-based intelligence;Science Advances;2023-10-06

同舟云学术

1.学者识别学者识别

2.学术分析学术分析

3.人才评估人才评估

"同舟云学术"是以全球学者为主线,采集、加工和组织学术论文而形成的新型学术文献查询和分析系统,可以对全球学者进行文献检索和人才价值评估。用户可以通过关注某些学科领域的顶尖人物而持续追踪该领域的学科进展和研究前沿。经过近期的数据扩容,当前同舟云学术共收录了国内外主流学术期刊6万余种,收集的期刊论文及会议论文总量共计约1.5亿篇,并以每天添加12000余篇中外论文的速度递增。我们也可以为用户提供个性化、定制化的学者数据。欢迎来电咨询!咨询电话:010-8811{复制后删除}0370

www.globalauthorid.com

TOP

Copyright © 2019-2024 北京同舟云网络信息技术有限公司
京公网安备11010802033243号  京ICP备18003416号-3