Author:
Buffoni Lorenzo,Civitelli Enrico,Giambagli Lorenzo,Chicchi Lorenzo,Fanelli Duccio
Abstract
AbstractTraining of neural networks can be reformulated in spectral space, by allowing eigenvalues and eigenvectors of the network to act as target of the optimization instead of the individual weights. Working in this setting, we show that the eigenvalues can be used to rank the nodes’ importance within the ensemble. Indeed, we will prove that sorting the nodes based on their associated eigenvalues, enables effective pre- and post-processing pruning strategies to yield massively compacted networks (in terms of the number of composing neurons) with virtually unchanged performance. The proposed methods are tested for different architectures, with just a single or multiple hidden layers, and against distinct classification tasks of general interest.
Publisher
Springer Science and Business Media LLC
Reference33 articles.
1. He, Y. et al. Amc: Automl for model compression and acceleration on mobile devices. In Proceedings of the European Conference on Computer Vision (ECCV), 784–800 (2018).
2. Sutton, R. S. & Barto, A. G. Reinforcement Learning: An Introduction (MIT press, ***, 2018).
3. Grigorescu, S., Trasnea, B., Cocias, T. & Macesanu, G. A survey of deep learning techniques for autonomous driving. J. Field Robot. 37, 362–386 (2020).
4. Goodfellow, I., Bengio, Y. & Courville, A. Deep Learning (MIT Press, 2016). http://www.deeplearningbook.org.
5. Giambagli, L., Buffoni, L., Carletti, T., Nocentini, W. & Fanelli, D. Machine learning in spectral domain. Nat. Commun. 12, 1–9 (2021).
Cited by
9 articles.
订阅此论文施引文献
订阅此论文施引文献，注册后可以免费订阅5篇论文的施引文献，订阅后可以查看论文全部施引文献