Channel Pruning for Accelerating Very Deep Neural Networks

TLDR

The authors propose a novel channel pruning technique to accelerate very deep convolutional neural networks. They employ an iterative two‑step algorithm that uses LASSO‑based channel selection followed by least‑squares reconstruction, extended to multi‑layer and multi‑branch architectures. The method achieves a 5× speed‑up on VGG‑16 with only a 0.3% error increase and delivers 2× speed‑ups on ResNet and Xception with 1.4% and 1.0% accuracy loss, while reducing accumulated error and improving compatibility across architectures.

Abstract

In this paper, we introduce a new channel pruning method to accelerate very deep convolutional neural networks. Given a trained CNN model, we propose an iterative two-step algorithm to effectively prune each layer, by a LASSO regression based channel selection and least square reconstruction. We further generalize this algorithm to multi-layer and multi-branch cases. Our method reduces the accumulated error and enhance the compatibility with various architectures. Our pruned VGG-16 achieves the state-of-the-art results by 5× speed-up along with only 0.3% increase of error. More importantly, our method is able to accelerate modern networks like ResNet, Xception and suffers only 1.4%, 1.0% accuracy loss under 2× speedup respectively, which is significant.