Not Too Hot, Not Too Cold: The Bundled-SVM is Just Right!

Abstract

The Support Vector Machine (SVM) typi-cally outperforms other algorithms on text classification problems, but requires train-ing time roughly quadratic in the number of training documents. In contrast, linear time algorithms like Naive Bayes have lower per-formance, but can easily handle huge training sets. In this paper, we describe a technique that creates a continuum of classifiers be-tween the SVM and a Naive Bayes like algo-rithm. Included in that continuum is a clas-sifier that approximates SVM performance with linear training time. Another classifier on this continuum can outperform the SVM, yielding a breakeven point that beats other published results on Reuters-21578. We give empirical and theoretical evidence that our hybrid approach successfully navigates the tradeoffs between speed and performance. 1. Introduction & Related