arXiv (Cornell University) · 2016 · 5.9K citations · 36 references
Open access
ComputingComputer EngineeringComputer ScienceComputer NetworksComputer VisionMachine LearningDeep LearningHardware AccelerationNeural NetworksPattern RecognitionScalable Machine LearningAccuracyCompressionModel Compression
Deep neural networks have traditionally prioritized accuracy, yet many architectures can reach the same performance, and smaller models offer benefits such as reduced communication, lower bandwidth for deployment, and suitability for memory‑constrained hardware. The authors introduce SqueezeNet to deliver these advantages. SqueezeNet is a compact architecture designed to match AlexNet accuracy while minimizing parameters. It attains AlexNet‑level ImageNet accuracy with 50× fewer parameters and can be compressed to under 0.5 MB, a 510× reduction. The architecture is publicly available at the provided URL.
Recent research on deep neural networks has focused primarily on improving accuracy. For a given accuracy level, it is typically possible to identify multiple DNN architectures that achieve that accuracy level. With equivalent accuracy, smaller DNN architectures offer at least three advantages: (1) Smaller DNNs require less communication across servers during distributed training. (2) Smaller DNNs require less bandwidth to export a new model from the cloud to an autonomous car. (3) Smaller DNNs are more feasible to deploy on FPGAs and other hardware with limited memory. To provide all of these advantages, we propose a small DNN architecture called SqueezeNet. SqueezeNet achieves AlexNet-level accuracy on ImageNet with 50x fewer parameters. Additionally, with model compression techniques we are able to compress SqueezeNet to less than 0.5MB (510x smaller than AlexNet). The SqueezeNet architecture is available for download here: this https URL
36
ImageNet: A large-scale hierarchical image database
Jia Deng, Wei Dong, Richard Socher et al. · 2009 IEEE Conference on Computer Vision and Pattern Recognition · 2009 · 61.8K citations
Dropout: a simple way to prevent neural networks from overfitting
Nitish Srivastava, Geoffrey E. Hinton, Alex Krizhevsky et al. · 2014 · 34.3K citations
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
Kaiming He, Xiangyu Zhang, Shaoqing Ren et al. · 2015 · 18.8K citations