arXiv (Cornell University) · 2017 · 378 citations · 10 references
ComputingComputer EngineeringComputer ScienceComputer SecurityMachine LearningUncertainty QuantificationDeep LearningNeural NetworksRepresentation LearningAdversarial Detection RobustnessAdversarial ExamplesAdversarial RobustnessBayesian Model UncertaintyBayesian UncertaintyDeep Image FeaturesDropoutModel Confidence Estimation
Deep neural networks (DNNs) are powerful nonlinear architectures that are known to be robust to random perturbations of the input. However, these models are vulnerable to adversarial perturbations--small input changes crafted explicitly to fool the model. In this paper, we ask whether a DNN can distinguish adversarial samples from their normal and noisy counterparts. We investigate model confidence on adversarial samples by looking at Bayesian uncertainty estimates, available in dropout neural networks, and by performing density estimation in the subspace of deep features learned by the model. The result is a method for implicit adversarial detection that is oblivious to the attack algorithm. We evaluate this method on a variety of standard datasets including MNIST and CIFAR-10 and show that it generalizes well across different architectures and attacks. Our findings report that 85-93% ROC-AUC can be achieved on a number of standard classification tasks with a negative class that consists of both normal and noisy samples.
10
Dropout: a simple way to prevent neural networks from overfitting
Nitish Srivastava, Geoffrey E. Hinton, Alex Krizhevsky et al. · 2014 · 34.3K citations
Backpropagation Applied to Handwritten Zip Code Recognition
Yann LeCun, Bernhard E. Boser, J. S. Denker et al. · Neural Computation · 1989 · 11.9K citations
The Limitations of Deep Learning in Adversarial Settings
Nicolas Papernot, Patrick McDaniel, Somesh Jha et al. · 2016 · 3.9K citations
Adversarial Examples in the Physical World
Alexey Kurakin, Ian Goodfellow, Samy Bengio · 2018 · 1.9K citations · Full text