2019 · 14 citations · 12 references
ComputingComputer EngineeringComputer ScienceComputer VisionMachine LearningMobile ComputingData CompressionDeep LearningImage CompressionLossless Data CompressionLossy Data CompressionNeural NetworksCompressionFiltersLayer PruningModel CompressionNeural Data CompressionPruningPruning Compression Trade-offsTraining Pruning
High performance of deep learning models typically comes at cost of considerable model size and computation time. These factors limit applicability for deployment on memory and battery constrained devices such as mobile phones or embedded systems. In this work, we propose a novel pruning technique that eliminates entire filters and neurons according to their relative L1-norm as compared to the rest of the network, yielding more compression and decreased parameters' redundancy. The resulting network is non-sparse, however, much more compact and requires no special infrastructure for deployment. We prove the viability of our method by achieving 97.4%, 86.1%, 47.8% and 53% compression of LeNet-5, VGG-16, ResNet-56 and ResNet-110 respectively, exceeding state-of-the-art compression results reported on VGG-16 and ResNet without losing any performance compared to the baseline. Our approach does not only exhibit good performance but is also easy to implement.
12
Deep Residual Learning for Image Recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren et al. · 2016 · 222.1K citations · Full text
Gradient-based learning applied to document recognition
Yann LeCun, Léon Bottou, Yoshua Bengio et al. · Proceedings of the IEEE · 1998 · 58.2K citations · Full text
Advances in neural information processing systems 7
Krzysztof J. Cios, Mark E. Shields · Neurocomputing · 1997 · 22.3K citations
NISP: Pruning Networks Using Neuron Importance Score Propagation
Ruichi Yu, Ang Li, Chun-Fu Chen et al. · 2018 · 851 citations