Affiliation:
1. State Key Laboratory of Industrial Control Technology, College of Control Science and Engineering, Zhejiang University, Hangzhou 310027, China
2. Key Laboratory of Collaborative Sensing and Autonomous Unmanned Systems of Zhejiang Province, Hangzhou 310027, China
Abstract
Semantic segmentation predicts dense pixel-wise semantic labels, which is crucial for autonomous environment perception systems. For applications on mobile devices, current research focuses on energy-efficient segmenters for both frame and event-based cameras. However, there is currently no artificial neural network (ANN) that can perform efficient segmentation on both types of images. This paper introduces spiking neural network (SNN, a bionic model that is energy-efficient when implemented on neuromorphic hardware) and develops a Spiking Context Guided Network (Spiking CGNet) with substantially lower energy consumption and comparable performance for both frame and event-based images. First, this paper proposes a spiking context guided block that can extract local features and context information with spike computations. On this basis, the directly-trained SCGNet-S and SCGNet-L are established for both frame and event-based images. Our method is verified on the frame-based dataset Cityscapes and the event-based dataset DDD17. On the Cityscapes dataset, SCGNet-S achieves comparable results to ANN CGNet with 4.85 × energy efficiency. On the DDD17 dataset, Spiking CGNet outperforms other spiking segmenters by a large margin.
Funder
STI 2030-Major Projects
Autonomous Intelligent Unmanned Systems
Subject
Molecular Medicine,Biomedical Engineering,Biochemistry,Biomaterials,Bioengineering,Biotechnology
Reference68 articles.
1. Cgnet: A light-weight context guided network for semantic segmentation;Wu;IEEE Trans. Image Process.,2020
2. SegFormer: Simple and efficient design for semantic segmentation with transformers;Xie;Adv. Neural Inf. Process. Syst.,2021
3. Isomorphic model-based initialization for convolutional neural networks;Zhang;J. Vis. Commun. Image Represent.,2022
4. Long, J., Shelhamer, E., and Darrell, T. (2015, January 7–12). Fully convolutional networks for semantic segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.
5. Binas, J., Neil, D., Liu, S.C., and Delbruck, T. (2017). DDD17: End-to-end DAVIS driving dataset. arXiv.
Cited by
2 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献