Mutation Integration Testing

Abstract

In integration testing, integrated software modules or components are evaluated as a whole to determine if they behave correctly. Mutation testing is recognized as one of the strongest approaches for evaluating the effectiveness of test suites, and it is important to generate effective mutants efficiently for integration tests. However, it is difficult to generate integration mutants that create an error state in one component with certain assurances that this error state will affect computations in some other components. Unfortunately, little research exists that addresses this big and important problem to improve the quality of integration test suites. In this paper, we propose a theory and a solution for generating mutants that specifically target integration tests. We formulate a fault model for integration bugs that uses static dataflow analysis to obtain information about how integrated components interact in an application. Integration mutants are generated by applying mutation operators to instructions that lie in dataflow paths among integrated components. We implemented our approach and evaluated it on five open-source applications. In comparison to muJava, our approach reduces the number of generated mutants by up to approximately 19 times with a strong power to determine inadequacies in integration test suites.