Are random forests truly the best classifiers

Abstract

The JMLR study Do we need hundreds of classifiers to solve real world classification problems? benchmarks 179 classifiers in 17 families on 121 data sets from UCI repository and claims that the random forest is clearly best family of classifier. In this response, we show that study's results are biased by lack of a held-out test set and exclusion of trials with errors. Further, study's own statistical tests indicate that random forests do not have significantly higher percent accuracy than support vector machines and neural networks, calling into question conclusion that random forests are best classifiers.