Cost and Coverage Metrics for Measuring the Effectiveness of Test Case Prioritization Techniques

Abstract

Regression testing is an important and also a very expensive activity in the software life cycle. To reduce the cost of regression testing, test cases are prioritized. One goal of test case prioritization technique is to increase a test suite’s rate of fault detection and to reduce the cost of regression testing. In his paper G. Rothermel [9] has provided a metric, Average Percentage of Fault Detected (APFD), for measuring rate of fault detection during prioritization. This metric assumes that all test cases and fault costs are uniform. In practice, test case and fault costs may vary, and in such cases the previous APFD metric can be unsatisfactory. This paper presents a metric for assessing the rate of fault detection of prioritized test cases, APFDc, that incorporates varying test cases and fault costs. We have also calculated other new metrics like Average Percentage of Statement Coverage (APSC), Average Percentage of Branch Coverage (APBC),Average Percentage of Loop Coverage (APLC) and Average Percentage of Condition Coverage (APCC) based on the coverage criterion for the various prioritization techniques performed. Test cases are executed using JUnit tool. Code cover tool is used to find code coverage information. Test case prioritization is performed based on coverage and cost information. By injecting mutation faults effectiveness of prioritization is measured. Finally, we have implemented all the metrics considering a few standard java programs.