Mutation analysis for SystemC designs at TLM

Abstract

Mutation analysis has been borrowed from the software testing domain as a technique for evaluating the quality of testbenches in validating digital systems. This paper presents a new method for applying mutation analysis on SystemC hardware designs at Transaction-Level Modeling (TLM). The method injects mutants by directly perturbing the SystemC code. Five key categories of mutation operators are implemented in order to speed up the analysis process. In the paper, a comparison of mutation analysis at two different abstraction levels - TLM and Register-Transfer Level (RTL), is carried out. The experiments show that mutation analysis is considerably faster at TLM than it is at RTL while achieving almost equal mutant coverage. Last but not least, TLM mutation analysis provides also more readable feedback for the engineer to improve the testbench. To the best of our knowledge this is the first method for mutation analysis directly working on uncompiled SystemC TLM code.