The design of a prototype mutation system for program testing.

Abstract

When testing software the major question which must always be addressed is a program is correct for a finite number of test cases, can we assume it is correct in general. Test data which possess this property is called Adequate test data, and, although adequate test data cannot in general be derived algorithmically, 1 several methods have recently emerged which allow one to gain confidence in one's test data's adequacy. Program mutation is a radically new approach to determining test data adequacy which holds promise of being a major breakthrough in the field of software testing. The concepts and philosophy of program mutation have been given elsewhere, 2 the following will merely present a brief introduction to the ideas underlying the system. Unlike previous work, program mutation assumes that competent programmers will produce programs which, if they are not correct, are almost correct. That is, if a program is not correct it is a mutant-it differs from a correct program by simple errors. Assuming this natural premise, a program P which is correct on test data T is subjected to a series of mutant operators to produce mutant programs which differ from P in very simple ways. The mutants are then executed on T. If all mutants give incorrect results then it is very likely that P is correct (i.e., T is adequate). On the other hand, if some mutants are correct on T then either: (1) the mutants are equivalent to P, or (2) the test data T is inadequate. In the latter case, T must be augmented by examining the non-equivalent mutants which are correct on T: a procedure which forces close examination of P with respect to the mutants. At first glance it would appear that if T is determined adequate by mutation analysis, then P might still contain some complex errors which are not explicitly mutants of P.