Predicting risk of software changes

TLDR

Reducing software failures is a major challenge, and historic change data suggests the model can be widely applied. We model the probability that a change causes a failure using predictors based on change size, diffusion, developer experience, and change type, building the model on historic data and implementing it as a web tool. Applying the model to 5ESS updates shows that change diffusion and developer experience are key predictors, and that predicting change quality enables better inspection, testing, and delivery decisions.

Abstract

Reducing the number of software failures is one of the most challenging problems of software production. We assume that software development proceeds as a series of changes and model the probability that a change to software will cause a failure. We use predictors based on the properties of a change itself. Such predictors include size in lines of code added, deleted, and unmodified; diffusion of the change and its component subchanges, as reflected in the number of files, modules, and subsystems touched, or changed; several measures of developer experience; and the type of change and its subchanges (fault fixes or new code). The model is built on historic information and is used to predict the risk of new changes. In this paper we apply the model to 5ESS® software updates and find that change diffusion and developer experience are essential to predicting failures. The predictive model is implemented as a Web-based tool to allow timely prediction of change quality. The ability to predict the quality of change enables us to make appropriate decisions regarding inspection, testing, and delivery. Historic information on software changes is recorded in many commercial software projects, suggesting that our results can be easily and widely applied in practice.