Column Pruning Beats Stratification in Effort Estimation

Abstract

Local calibration combined with stratification, also known as row pruning, is a common technique used by cost estimation professionals to improve model performance. The results presented in this paper raise several serious questions concerning the benefits of row pruning for improving effort estimation indicating the need to rethink standard practice. Firstly, the mean size of improvements from row pruning appears to be relatively small compared to the size of the standard deviations in effort estimation data. Secondly, the advantages of row pruning especially for the purposes of deleting spurious outliers can be achieved using column pruning much more effectively. Hence, we advise against row pruning and advocate column pruning instead.