2018 · 24 citations · 21 references
ComputingComputer ScienceComputer VisionFeature EngineeringMachine LearningDeep LearningNeural NetworksOptimization For Machine LearningPruning StrategiesRepresentation LearningClusteringDeep Learning AcceleratorsLayer PruningLayer-wise PruningOnline ClusteringPruning During Pre-training
Pruning filters is an effective method for accelerating deep neural networks (DNNs), but most existing approaches prune filters on a pre-trained network directly which limits in acceleration. Although each filter has its own effect in DNNs, but if two filters are same with each other, we could prune one safely. In this paper, we add an extra cluster loss term in the loss function which can force filters in each cluster to be similar online. After training, we keep one filter in each cluster and prune others and fine-tune the pruned network to compensate the loss. Particularly, the clusters in every layer can be defined firstly which is effective for pruning DNNs within residual blocks. Extensive experiments on CIFAR10 and CIFR100 benchmarks demonstrate the competitive performance of our proposed filter pruning method.
21
Distilling the Knowledge in a Neural Network
Geoffrey E. Hinton, Oriol Vinyals · arXiv (Cornell University) · 2015 · 14K citations · Full text
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
Andrew Howard, Menglong Zhu, Bo Chen et al. · arXiv (Cornell University) · 2017 · 9.9K citations · Full text
TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
Martı́n Abadi, Ashish Agarwal, Paul Barham et al. · arXiv (Cornell University) · 2016 · 9.8K citations · Full text
Sergey Zagoruyko, Nikos Komodakis · 2016 · 6K citations
Yann LeCun, John S. Denker, Sara A. Solla · 1989 · 2.6K citations