The AETG system: an approach to testing based on combinatorial design

TLDR

Testing relies on system parameters such as configuration settings, user inputs, and external events, and the size of a combinatorial test set grows logarithmically with the number of parameters, allowing models with dozens of parameters. The paper proposes a new testing approach that uses combinatorial designs to generate tests covering pairwise, triple, or n‑way parameter combinations. The AETG system implements new combinatorial algorithms that generate test sets covering all valid n‑way parameter combinations for unit, system, and interoperability testing. The system produced both high‑level test plans and detailed test cases, and in several applications it significantly reduced test‑plan development costs.

Abstract

This paper describes a new approach to testing that uses combinatorial designs to generate tests that cover the pairwise, triple, or n-way combinations of a system's test parameters. These are the parameters that determine the system's test scenarios. Examples are system configuration parameters, user inputs and other external events. We implemented this new method in the AETG system. The AETG system uses new combinatorial algorithms to generate test sets that cover all valid n-way parameter combinations. The size of an AETG test set grows logarithmically in the number of test parameters. This allows testers to define test models with dozens of parameters. The AETG system is used in a variety of applications for unit, system, and interoperability testing. It has generated both high-level test plans and detailed test cases. In several applications, it greatly reduced the cost of test plan development.