2008 · 200 citations · 8 references
ComputingComputer ScienceAlgorithmsMachine LearningStatistical LearningBiasBias-variance TradeoffClassificationDecision TreesProbabilistic Machine LearningStatistical InferenceDecision Tree ClassificationDecision-tree AnalysisMachine Learning Bias CorrectionOverfitting In TreesProbabilistic Decision TreesRandomized Decision TreesStatistical Bias CorrectionTree Algorithms
The term "bias" is widely used---and with different meanings---in the fields of machine learning and statistics. This paper clarifies the uses of this term and shows how to measure and visualize the statistical bias and variance of learning algorithms. Statistical bias and variance can be applied to diagnose problems with machine learning bias, and the paper shows four examples of this. Finally, the paper discusses methods of reducing bias and variance. Methods based on voting can reduce variance, and the paper compares Breiman's bagging method and our own tree randomization method for voting decision trees. Both methods uniformly improve performance on data sets from the Irvine repository. Tree randomization yields perfect performance on the Letter Recognition task. A weighted nearest neighbor algorithm based on the infinite bootstrap is also introduced. In general, decision tree algorithms have moderate-to-high variance, so an important implication of this work is that variance---rat...
8
Leo Breiman · Machine Learning · 1996 · 17K citations · Full text
The Strength of Weak Learnability
Robert E. Schapire · Machine Learning · 1990 · 3.3K citations · Full text
The Weighted Majority Algorithm
N. Littlestone, Manfred K. Warmuth · Information and Computation · 1994 · 2K citations
The Need for Biases in Learning Generalizations
Tom M. Mitchell · BMC Nursing · 2007 · 530 citations · Full text