A fortran language system for mutation‐based software testing

TLDR

Mutation analysis, which evaluates program quality by executing many slightly altered versions, is a powerful testing technique and forms the basis of the Mothra integrated testing environment. This paper examines the specific requirements of a language system for mutation‑based testing and how they shaped the design of Mothra’s Fortran 77 implementation. Mothra’s system translates programs into an intermediate language that can be executed by an interpreter, enabling the creation and execution of mutated versions within the same environment. The techniques developed for Mothra’s language system are broadly applicable to other program analysis tools, such as debuggers, measurement tools, and development environments. Appendices provide a complete description of the intermediate language and the mutation operators used by Mothra.

Abstract

Abstract Mutation analysis is a powerful technique for testing software systems. The Mothra software testing project uses mutation analysis as the basis for an integrated software testing environment. Mutation analysis requires executing many slightly differing versions of the same program to evaluate the quality of the data used to test the program. The current version of Mothra includes a complete language system that translates a program to be tested into intermediate code so that it and its mutated versions can be executed by an interpreter. In this paper, we discuss some of the unique requirements of a language system used in a mutation‐based testing environment. We then describe how these requirements affected the design and implementation of the Fortran 77 version of the Mothra system. We also describe the intermediate language used by Mothra and the features of the language system that are needed for software testing. The appendices contain a full description of the intermediate language and the mutation operators used by Mothra. The design and implementation techniques that were developed for Mothra are applicable for constructing not just software testing systems, but any type of program analysis system or language system for a special‐purpose application. In particular, we discuss decisions made and techniques developed by the Mothra team that can be useful in such applications as debuggers, program measurement tools, software development environments and other types of program analysis systems.