An empirical evaluation of supervised learning in high dimensions

Abstract

In this paper we perform an empirical evaluation of supervised learning on high-dimensional data. We evaluate perfor-mance on three metrics: accuracy, AUC, and squared loss and study the eect of increas-ing dimensionality on the performance of the learning algorithms. Our ndings are con-sistent with previous studies for problems of relatively low dimension, but suggest that as dimensionality increases the relative perfor-mance of the learning algorithms changes. To our surprise, the method that performs consistently well across all dimensions is ran-dom forests, followed by neural nets, boosted trees, and SVMs. 1.