Experiments on the effectiveness of dataflow- and control-flow-based test adequacy criteria

TLDR

The study evaluates how effective two code‑based test adequacy criteria are at selecting fault‑detecting test cases. The authors applied all‑edges and all‑DU coverage to 130 seeded faulty versions of seven moderate‑size programs, generating thousands of test sets per program to analyze fault detection versus coverage. Test sets with >90% coverage detected faults more reliably than random sets, with additional gains up to 100% coverage, but 100% coverage alone is not a definitive indicator, and control‑flow and dataflow criteria complement each other. No additional metadata available.

Abstract

This paper reports an experimental study investigating the effectiveness of two code-based test adequacy criteria for identifying sets of test cases that detect faults. The all-edges and all-DUs (modified all-uses) coverage criteria were applied to 130 faulty program versions derived from seven moderate size base programs by seeding realistic faults. We generated several thousand test sets for each faulty program and examined the relationship between fault detection and coverage. Within the limited domain of our experiments, test sets achieving coverage levels over 90% usually showed significantly better fault detection than randomly chosen test sets of the same size. In addition, significant improvements in the effectiveness of coverage-based tests usually occurred as coverage increased from 90% to 100%. However the results also indicate that 100% code coverage alone is not a reliable indicator of the effectiveness of a test set. We also found that tests based respectively on control-flow and dataflow criteria are frequency complementary in their effectiveness. >