Using Machine Learning to Predict Project Effort: Empirical Case Studies in Data-Starved Domains

Abstract

Ideally, software engineering should be able to use machine learning to control or significantly decrease the costs associated with building software. In reality, there are very few examples of applying such applications early in the software life cycle. One reason for the scarcity of examples is the lack of empirical data in the software engineering discipline. This dilemma is quite evident when constructing models to predict project effort. This raises the question of "How to generate sufficient amounts of data when it is sparse?" One approach is to assess projects from a bottomup perspective. This approach uses estimates gathered from products in predicting project effort. This paper conducts a set of machine learning experiments with software cost estimation data from two separate organizations. These experiments explore the possibility of performing project estimating from a bottom-up perspective and characterize predictive potential within two different organizations. The results are statistically assessed and a process is proposed for applying the described techniques.