Author:
Davidson Simon,Furber Steve B.
Abstract
Despite the success of Deep Neural Networks—a type of Artificial Neural Network (ANN)—in problem domains such as image recognition and speech processing, the energy and processing demands during both training and deployment are growing at an unsustainable rate in the push for greater accuracy. There is a temptation to look for radical new approaches to these applications, and one such approach is the notion that replacing the abstract neuron used in most deep networks with a more biologically-plausible spiking neuron might lead to savings in both energy and resource cost. The most common spiking networks use rate-coded neurons for which a simple translation from a pre-trained ANN to an equivalent spike-based network (SNN) is readily achievable. But does the spike-based network offer an improvement of energy efficiency over the original deep network? In this work, we consider the digital implementations of the core steps in an ANN and the equivalent steps in a rate-coded spiking neural network. We establish a simple method of assessing the relative advantages of rate-based spike encoding over a conventional ANN model. Assuming identical underlying silicon technology we show that most rate-coded spiking network implementations will not be more energy or resource efficient than the original ANN, concluding that more imaginative uses of spikes are required to displace conventional ANNs as the dominant computing framework for neural computation.
Funder
H2020 Research Infrastructures
Huawei Technologies
Reference31 articles.
1. AbadiM.
AgarwalA.
BarhamP.
BrevdoE.
ChenZ.
CitroC.
TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems2015
2. A state-of-the-art survey on deep learning theory and architectures;Alom;Electronics,2019
3. Neural correlations, population coding and computation;Averbeck;Nat. Rev. Neurosci,2006
4. “Training deep networks for facial expression recognition with crowd-sourced label distribution,”;Barsoum,2016
5. Language models are few-shot learners;Brown,2020
Cited by
49 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献