Affiliation:
1. EEE Department, Imperial College London, London, UK
Abstract
We consider efficiency in the implementation of deep neural networks. Hardware accelerators are gaining interest as machine learning becomes one of the drivers of high-performance computing. In these accelerators, the directed graph describing a neural network can be implemented as a directed graph describing a Boolean circuit. We make this observation precise, leading naturally to an understanding of practical neural networks as discrete functions, and show that the so-called
binarized neural networks
are functionally complete. In general, our results suggest that it is valuable to consider
Boolean circuits as neural networks
, leading to the question of which circuit topologies are promising. We argue that continuity is central to generalization in learning, explore the interaction between data coding, network topology, and node functionality for continuity and pose some open questions for future research. As a first step to bridging the gap between continuous and Boolean views of neural network accelerators, we present some recent results from our work on LUTNet, a novel Field-Programmable Gate Array inference approach. Finally, we conclude with additional possible fruitful avenues for research bridging the continuous and discrete views of neural networks.
This article is part of a discussion meeting issue ‘Numerical algorithms for high-performance computational science’.
Funder
Imagination Technologies
Engineering and Physical Sciences Research Council
Royal Academy of Engineering
Subject
General Physics and Astronomy,General Engineering,General Mathematics
Reference49 articles.
1. Deep Neural Network Approximation for Custom Hardware
2. Scheinberg K. 2016 Evolution of randomness in optimization methods for supervised machine learning. SIAG/OPT views and news (ed. S Wild) vol. 24 pp. 1–7. http://wiki.siam.org/siag-op/index.php/View_and_News.
3. LeCun Y. 1989 Generalization and network design strategies. University of Toronto Technical Report. CRG-TR-89-4.
Cited by
1 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献