2018 · 17 citations · 16 references
ComputingComputer ScienceComputer VisionMachine LearningData CompressionDeep LearningImage CompressionLossless Data CompressionLossy Data CompressionOptical FlowAverage PoolingConvolutional Neural NetworksDeep Learning-based Image CompressionFeature PoolingGlobal PoolingMax PoolingModel Compression
Deep neural networks are powerful, but using these networks is both memory and time consuming for their numerous parameters and large amounts of computation. There are many studies in compressing the models. On the parameter-level, one of the compressing methods is to spend lots of time performing the iterative process consisting of pruning weights and fine-tuning the models. On the bit-level, many studies use quantization to cut down on the number of need bits. Hence, we propose an efficient strategy to compress on the layers which are computation or memory consuming. We compress the model by adding the global average pooling, iteratively pruning on the filters with proposed order-deciding scheme to prune more efficiently, applying the truncated SVD to the fully-connected layer, and performing the two-stage quantization. Experiments on the VGG16 model show that we can reach a 60.9× compression ratio in off-line storage with about 0.848% and 0.1378% loss of accuracy on the top-1 and top-5 classification results with the validation dataset of ILSVRC2012.
16
ImageNet Large Scale Visual Recognition Challenge
Olga Russakovsky, Jia Deng, Hao Su et al. · International Journal of Computer Vision · 2015 · 40.2K citations
Yangqing Jia, Evan Shelhamer, Jeff Donahue et al. · 2014 · 11.2K citations
Min Lin, Qiang Chen, Shuicheng Yan · arXiv (Cornell University) · 2013 · 2K citations · Full text