Filter pruning via feature map clustering
-
Published:2023-07-20
Issue:4
Volume:27
Page:911-933
-
ISSN:1088-467X
-
Container-title:Intelligent Data Analysis
-
language:
-
Short-container-title:IDA
Author:
Li Wei1, He Yongxing1, Zhang Xiaoyu1, Tang Yongchuan12
Affiliation:
1. College of Computer Science, Zhejiang University, Hangzhou, Zhejiang, China 2. Alibaba-Zhejiang University Joint Research Institute of Frontier Technologise, Hangzhou, Zhejiang, China
Abstract
With the help of network compression algorithms, deep neural networks can be applied on low-power embedded systems and mobile devices such as drones, satellites, and smartphones. Filter pruning is a sub-direction of network compression research, which reduces memory and computational consumption by reducing the number of parameters of model filters. Previous works utilized the “more-simple-less-important” criterion for pruning filters. That is, filters with the smaller norm or more sparse weights in the network are preferentially pruned. In this paper, we found that feature maps are not fully positively correlated with the sparsity of filter weights by observing the visualization of feature maps and the corresponding filters. Hence, we came up with the idea that the priority of filter pruning should be determined by redundancy rather than sparsity. The redundancy of a filter is the measure of whether the output of the filter is repeated with other filters. Based on this, we defined a criterion called redundancy index to rank the filters and introduced it into our filter pruning strategy. Extensive experiments demonstrate the effectiveness of our approach on different model architectures, including VGGNet, GoogleNet, DenseNet, and ResNet. The models compressed with our strategy surpass the state-of-the-art in terms of Floating Point Operations Per Second (FLOPs), parameters reduction, and classification accuracy.
Subject
Artificial Intelligence,Computer Vision and Pattern Recognition,Theoretical Computer Science
Reference38 articles.
1. F. Han, L. Yu and Y. Jiang, Computer-aided diagnosis system of lung carcinoma using Convolutional Neural Networks, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020, pp. 690–691. 2. C. Chen, A. Seff, A. Kornhauser and J. Xiao, Deepdriving: Learning affordance for direct perception in autonomous driving, in: Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2015, pp. 2722–2730. 3. D. Ye, Z. Liu, M. Sun, B. Shi, P. Zhao, H. Wu, H. Yu, S. Yang, X. Wu, Q. Guo et al., Mastering complex control in moba games with deep reinforcement learning, in: Proceedings of the Association for the Advancement of Artificial Intelligence (AAAI), Vol. 34, 2020, pp. 6672–6679. 4. W. Chen, J. Wilson, S. Tyree, K. Weinberger and Y. Chen, Compressing neural networks with the hashing trick, in: Proceedings of the International Conference on Machine Learning (ICML), PMLR, 2015, pp. 2285–2294. 5. H. Li, A. Kadav, I. Durdanovic, H. Samet and H.P. Graf, Pruning filters for efficient convnets, in: Proceedings of the International Conference on Learning Representations (ICLR), 2017.
|
|