arXiv (Cornell University) · 2018 · 143 citations · 24 references
ComputingComputer ScienceComputer VisionMachine LearningMachine VisionDeep LearningImage ClassificationNeural NetworksObject DetectionRepresentation LearningTransfer LearningAblation ProcessesClassification AccuracyDeep Learning-based ClassificationFeature TransferModel Accuracy Assessment
Recent advances in image classification are largely driven by training procedure refinements such as data augmentation and optimization, yet most of these refinements are only briefly mentioned or hidden in source code. This work aims to systematically evaluate the impact of these refinements on model accuracy and demonstrate that improved classification accuracy translates to better transfer learning performance. The authors conduct an ablation study of a collection of training refinements to measure their effect on final model accuracy. Combining the refinements yields significant accuracy gains, raising ResNet‑50’s ImageNet top‑1 accuracy from 75.3 % to 79.29 %.
Much of the recent progress made in image classification research can be credited to training procedure refinements, such as changes in data augmentations and optimization methods. In the literature, however, most refinements are either briefly mentioned as implementation details or only visible in source code. In this paper, we will examine a collection of such refinements and empirically evaluate their impact on the final model accuracy through ablation study. We will show that, by combining these refinements together, we are able to improve various CNN models significantly. For example, we raise ResNet-50's top-1 validation accuracy from 75.3% to 79.29% on ImageNet. We will also demonstrate that improvement on image classification accuracy leads to better transfer learning performance in other application domains such as object detection and semantic segmentation.
24
ImageNet Large Scale Visual Recognition Challenge
Olga Russakovsky, Jia Deng, Hao Su et al. · International Journal of Computer Vision · 2015 · 40.2K citations
Squeeze-and-Excitation Networks
Distilling the Knowledge in a Neural Network
Geoffrey E. Hinton, Oriol Vinyals · arXiv (Cornell University) · 2015 · 14K citations · Full text
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
Andrew Howard, Menglong Zhu, Bo Chen et al. · arXiv (Cornell University) · 2017 · 9.9K citations · Full text