2000 · 145 citations · 14 references
ComputingComputer ScienceData ScienceMachine LearningStatistical LearningStatisticsBayesian InferenceBayesian StatisticsEnsemble LearningOverfittingProbabilistic Machine LearningStatistical InferenceBayesian ClassifiersEnsemble AveragingEnsemble Learning TheoryModel AveragingStatistical Learning Theory
Although Bayesian model averaging is theoretically the optimal method for combining learned models, it has seen very little use in machine learning. In this paper we study its application to combining rule sets, and compare it with bagging and partitioning, two popular but more ad hoc alternatives. Our experiments show that, surprisingly, Bayesian model averaging's error rates are consistently higher than the other methods'. Further investigation shows this to be due to a marked tendency to overfit on the part of Bayesian model averaging, contradicting previous beliefs that it solves (or avoids) the overfitting problem.
14
Leo Breiman · Machine Learning · 1996 · 17K citations · Full text
UCI Repository of machine learning databases
Catherine Blake · Medical Entomology and Zoology · 1998 · 10.5K citations
Experiments with a new boosting algorithm
Yoav Freund, Robert E. Schapire · 1996 · 7.6K citations
J. R. Quinlan · National Conference on Artificial Intelligence · 1996 · 1.3K citations