Affiliation:
1. State Key Laboratory of Industrial Control Technology, College of Control Science and Engineering, Zhejiang University, Hangzhou 310027, China
2. Key Laboratory of Collaborative Sensing and Autonomous Unmanned Systems of Zhejiang Province, Hangzhou 310027, China
Abstract
Spiking neural networks (SNNs) are widely recognized for their biomimetic and efficient computing features. They utilize spikes to encode and transmit information. Despite the many advantages of SNNs, they suffer from the problems of low accuracy and large inference latency, which are, respectively, caused by the direct training and conversion from artificial neural network (ANN) training methods. Aiming to address these limitations, we propose a novel training pipeline (called IDSNN) based on parameter initialization and knowledge distillation, using ANN as a parameter source and teacher. IDSNN maximizes the knowledge extracted from ANNs and achieves competitive top-1 accuracy for CIFAR10 (94.22%) and CIFAR100 (75.41%) with low latency. More importantly, it can achieve 14× faster convergence speed than directly training SNNs under limited training resources, which demonstrates its practical value in applications.
Subject
Molecular Medicine,Biomedical Engineering,Biochemistry,Biomaterials,Bioengineering,Biotechnology
Reference31 articles.
1. Diverse synaptic plasticity mechanisms orchestrated to form and retrieve memories in spiking neural networks;Zenke;Nat. Commun.,2015
2. Snn-rat: Robustness-enhanced spiking neural network through regularized adversarial training;Ding;Adv. Neural Inf. Process. Syst.,2022
3. Two types of asynchronous activity in networks of excitatory and inhibitory spiking neurons;Ostojic;Nat. Neurosci.,2014
4. Surrogate gradient learning in spiking neural networks: Bringing the power of gradient-based optimization to spiking neural networks;Neftci;IEEE Signal Process. Mag.,2019
5. Bu, T., Fang, W., Ding, J., Dai, P., Yu, Z., and Huang, T. (2023). Optimal ANN-SNN conversion for high-accuracy and ultra-low-latency spiking neural networks. arXiv.
Cited by
3 articles.
订阅此论文施引文献
订阅此论文施引文献，注册后可以免费订阅5篇论文的施引文献，订阅后可以查看论文全部施引文献