Test case prioritization based on data reuse an experimental study

Abstract

The order in which tests are executed can significantly impact the total test execution time. In this paper, we evaluate two test prioritization techniques (manual and automatic) in the context of mobile phone testing. The manual technique produces test sequences created by test experts, while the automatic one generates sequences mechanically based on the permutation of the tests. Both techniques take into account a data reuse: the more the data is reused among tests, the faster the sequence is executed. In order to evaluate the benefits of these two techniques, we carried out an experiment with 8 testers and 2 test suites arranged in a 2times2 Latin square design replicated four times. The automatic technique reduced approximately 25% of the data generation time and 13.5% of the execution time. The automatic technique is clearly better than the manual one with respect to the generation of sequences. Our experiment showed that the automatic technique also generates sequences whose execution is faster than those created manually by test experts.