2020 · 611 citations · 61 references
ComputingComputer ScienceArtificial IntelligenceComputer VisionMachine LearningDeep LearningExplainable Machine LearningImage ClassificationNeural NetworksPattern RecognitionRepresentation LearningAdversarial ExamplesAdversarial RobustnessConvolutional Neural NetworksDeep Image FeaturesDeep Learning-based ClassificationFrequency Spectrum AnalysisHeuristicsHigh-frequency Modeling
We investigate the relationship between the frequency spectrum of image data and the generalization behavior of convolutional neural networks (CNN). We first notice CNN's ability in capturing the high-frequency components of images. These high-frequency components are almost imperceptible to a human. Thus the observation leads to multiple hypotheses that are related to the generalization behaviors of CNN, including a potential explanation for adversarial examples, a discussion of CNN's trade-off between robustness and accuracy, and some evidence in understanding training heuristics.
61
Deep Residual Learning for Image Recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren et al. · 2016 · 222.1K citations · Full text
Adam: A Method for Stochastic Optimization
Diederik P. Kingma, Jimmy Ba · UvA-DARE (University of Amsterdam) · 2014 · 84.8K citations · Full text
ImageNet: A large-scale hierarchical image database
Jia Deng, Wei Dong, Richard Socher et al. · 2009 IEEE Conference on Computer Vision and Pattern Recognition · 2009 · 61.8K citations