Could We Predict the Result of a Continuous Integration Build? An Empirical Study

Abstract

Software build integrates modules developed and maintained by different developers in parallel, tests the result of integration, and serves as a crucial step in cooperatiive software development. Predicting the result of build has drawn the interest of academia and industry. In spite of many previous researches, the generalizability of build failure prediction over a wide range of open-source projects remains unclear.In this paper, we used 9 classifiers to construct prediction models and investigated the performance of both cross-validation and on-line predictions on 126 open-source projects available on TravisTorrent with nearly 300,000 build records. We found that for most projects, (a) the prediction performance in cross-validation scenario is pretty well (especial under AUC); (b) when it comes to on-line scenario, the prediction performance falls to a fairly low level.