Abstract
The continued success of Deep Neural Networks (DNNs) in classification tasks has sparked a trend of accelerating their execution with specialized hardware. While published designs easily give an order of magnitude improvement over general-purpose hardware, few look beyond an initial implementation. This paper presents Minerva, a highly automated co-design approach across the algorithm, architecture, and circuit levels to optimize DNN hardware accelerators. Compared to an established fixed-point accelerator baseline, we show that fine-grained, heterogeneous datatype optimization reduces power by 1.5×; aggressive, inline predication and pruning of small activity values further reduces power by 2.0×; and active hardware fault detection coupled with domain-aware error mitigation eliminates an additional 2.7× through lowering SRAM voltages. Across five datasets, these optimizations provide a collective average of 8.1× power reduction over an accelerator baseline without compromising DNN model accuracy. Minerva enables highly accurate, ultra-low power DNN accelerators (in the range of tens of milliwatts), making it feasible to deploy DNNs in power-constrained IoT and mobile devices.
Publisher
Association for Computing Machinery (ACM)
Reference64 articles.
1. Multilayer feedforward networks are universal approximators
2. Deep Neural Networks for Acoustic Modeling in Speech Recognition: The Shared Views of Four Research Groups
3. A. Hannun C. Case J. Casper B. Catanzaro G. Diamos E. Elsen R. Prenger S. Satheesh S. Sengupta A. Coates and A. Y. Ng "Deep speech: Scaling up end-to-end speech recognition " arXiv:1412.5567 {cs.CL} 2014. A. Hannun C. Case J. Casper B. Catanzaro G. Diamos E. Elsen R. Prenger S. Satheesh S. Sengupta A. Coates and A. Y. Ng "Deep speech: Scaling up end-to-end speech recognition " arXiv:1412.5567 {cs.CL} 2014.
4. Imagenet classification with deep convolutional neural networks;Krizhevsky A.;Advances in Neural Information Processing Systems,2012
5. A unified architecture for natural language processing
Cited by
102 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献