Empirical studies of test‐suite reduction

TLDR

Test‑suite reduction aims to lower maintenance costs by removing redundant tests, but may compromise fault detection; prior work showed size reduction without major loss in fault detection for specific programs, yet generalization requires further study. This paper reports on the design and execution of additional studies examining the costs and benefits of test‑suite reduction and the factors influencing them. The authors conducted empirical studies to assess the costs and benefits of test‑suite reduction and the factors that influence them. Results reveal that test‑suite reduction can severely compromise fault‑detection capabilities, contrary to earlier findings. Copyright © 2002 John Wiley & Sons, Ltd.

Abstract

Abstract Test‐suite reduction techniques attempt to reduce the costs of saving and reusing test cases during software maintenance by eliminating redundant test cases from test suites. A potential drawback of these techniques is that reducing the size of a test suite might reduce its ability to reveal faults in the software. Previous studies have suggested that test‐suite reduction techniques can reduce test‐suite size without significantly reducing the fault‐detection capabilities of test suites. These studies, however, involved particular programs and types of test suites, and to begin to generalize their results, further work is needed. This paper reports on the design and execution of additional studies, examining the costs and benefits of test‐suite reduction, and the factors that influence these costs and benefits. In contrast to previous studies, results of these studies reveal that the fault‐detection capabilities of test suites can be severely compromised by test‐suite reduction. Copyright © 2002 John Wiley & Sons, Ltd.