An empirical comparison of supervised learning algorithms

TLDR

Supervised learning methods have proliferated over the last decade, yet the most recent comprehensive empirical evaluation was the Statlog Project of the early 1990s. The study aims to conduct a large-scale empirical comparison of ten supervised learning methods, including SVMs, neural nets, logistic regression, naive Bayes, memory-based learning, random forests, decision trees, bagged trees, boosted trees, and boosted stumps. The authors evaluate these methods across a variety of performance criteria, assess the effect of calibrating models with Platt Scaling and Isotonic Regression, and examine how calibration influences performance.

Abstract

A number of supervised learning methods have been introduced in the last decade. Unfortunately, the last comprehensive empirical evaluation of supervised learning was the Statlog Project in the early 90's. We present a large-scale empirical comparison between ten supervised learning methods: SVMs, neural nets, logistic regression, naive bayes, memory-based learning, random forests, decision trees, bagged trees, boosted trees, and boosted stumps. We also examine the effect that calibrating the models via Platt Scaling and Isotonic Regression has on their performance. An important aspect of our study is the use of a variety of performance criteria to evaluate the learning methods.