Reducing Variance of Committee Prediction with Resampling Techniques

Abstract

Algorithms for reducing variance in neural network prediction using committee and resampling techniques bootstrap and cross-validation are presented. Their effectiveness is tested on data sets with different levels of noise and on medical diagnosis data sets. The methods are most effective when the noise level in the data is high or the size of the learning set is small, which consequently produces high variance. The algorithms will not be of much help in cases where the error of prediction is mainly due to bias. An increase in bias is observed due to smaller effective learning size in the bootstrap and crossvalidation committee. The impact of increased bias on the performance ranges from negligible to completely undermining the benefit of reducing the variance.