Adaptive Random Test Case Prioritization

TLDR

Regression testing ensures that program changes do not introduce unintended faults, and rearranging test case order can improve effectiveness, yet random ordering is often viewed as ineffective; adaptive random testing has emerged as a promising alternative. The study proposes a new family of coverage‑based adaptive random testing techniques. The authors empirically demonstrate that these techniques are statistically superior to random‑testing‑based approaches in fault detection. One ART prioritization technique consistently matches the performance of top coverage‑based methods while requiring much less time.

Abstract

Regression testing assures changed programs against unintended amendments. Rearranging the execution order of test cases is a key idea to improve their effectiveness. Paradoxically, many test case prioritization techniques resolve tie cases using the random selection approach, and yet random ordering of test cases has been considered as ineffective. Existing unit testing research unveils that adaptive random testing (ART) is a promising candidate that may replace random testing (RT). In this paper, we not only propose a new family of coverage-based ART techniques, but also show empirically that they are statistically superior to the RT-based technique in detecting faults. Furthermore, one of the ART prioritization techniques is consistently comparable to some of the best coverage-based prioritization techniques (namely, the "additional" techniques) and yet involves much less time cost.