On the economics of requirements-based test case prioritization

Abstract

Software testing is a strenuous and expensive process. At least 50% of the total software cost is spent on testing activities [12]. Companies are often faced with time and resource constraints that limit their ability to effectively complete testing efforts. Companies generally save suites for reuse; test suite reuse accounts for almost half of the maintenance cost [9]. As the product goes thru several versions, executing all the test cases in a test suite can be expensive [9]. Prioritization of test cases can be cost effective when the time allocated to complete testing is limited [9]. Test case prioritization (TCP) involves the explicit planning of the execution of test cases in a specific order and is shown to improve the rate of fault detection [3, 9]. The current software TCP techniques are primarily coverage-based (statement, branch or other coverage) [3,9]. Coverage-based white-box prioritization techniques are most applicable for regression testing at the unit level and are harder to apply on complex systems [2]. These techniques require testers to read and understand the code, which can be time consuming [2], and may assume that all faults are equally severe.