Mutation testing of memory-related operators

Abstract

Though mutation operators have been designed for a wide range of programming languages in the last three decades, only a few operators are able to simulate memory faults. This paper introduces 9 Memory Mutation Operators targeting common memory faults. We report the results of an empirical study using 16 open source programs, which come with well designed unit test suites. We find only 44% of the new memory mutants introduced are captured by the traditional strong mutation killing criterion. We thus further introduce two new killing criteria, the Memory Fault Detection and the Control Flow Deviation killing criteria to augment the traditional strong mutation testing criterion. Our results show that the two new killing criteria are more effective at detecting memory mutants, killing between 10% and 75% of those mutants left unkilled by the traditional criterion.