Simple and Scalable Predictive Uncertainty Estimation using Deep\n Ensembles

Abstract

Deep neural networks (NNs) are powerful black box predictors that have\nrecently achieved impressive performance on a wide spectrum of tasks.\nQuantifying predictive uncertainty in NNs is a challenging and yet unsolved\nproblem. Bayesian NNs, which learn a distribution over weights, are currently\nthe state-of-the-art for estimating predictive uncertainty; however these\nrequire significant modifications to the training procedure and are\ncomputationally expensive compared to standard (non-Bayesian) NNs. We propose\nan alternative to Bayesian NNs that is simple to implement, readily\nparallelizable, requires very little hyperparameter tuning, and yields high\nquality predictive uncertainty estimates. Through a series of experiments on\nclassification and regression benchmarks, we demonstrate that our method\nproduces well-calibrated uncertainty estimates which are as good or better than\napproximate Bayesian NNs. To assess robustness to dataset shift, we evaluate\nthe predictive uncertainty on test examples from known and unknown\ndistributions, and show that our method is able to express higher uncertainty\non out-of-distribution examples. We demonstrate the scalability of our method\nby evaluating predictive uncertainty estimates on ImageNet.\n