A geometric approach for accelerating neural networks designed for classification problems

Abstract

AbstractThis paper proposes a geometric-based technique for compressing convolutional neural networks to accelerate computations and improve generalization by eliminating non-informative components. The technique utilizes a geometric index called separation index to evaluate the functionality of network elements such as layers and filters. By applying this index along with center-based separation index, a systematic algorithm is proposed that optimally compresses convolutional and fully connected layers. The algorithm excludes layers with low performance, selects the best subset of filters in the filtering layers, and tunes the parameters of fully connected layers using center-based separation index. An illustrative example of classifying CIFAR-10 dataset is presented to explain the algorithm step-by-step. The proposed method achieves impressive pruning results on networks trained by CIFAR-10 and ImageNet datasets, with 87.5%, 77.6%, and 78.8% of VGG16, GoogLeNet, and DenseNet parameters pruned, respectively. Comparisons with state-of-the-art works are provided to demonstrate the effectiveness of the proposed method.