A3Test: Assertion-Augmented Automated Test case generation

Abstract

Test case generation is a critical yet challenging task in software development. Recently, AthenaTest – a Deep Learning (DL) approach for generating unit test cases has been proposed. However, our revisiting study reveals that AthenaTest can generate less than one-fifth of the test cases correctly, due to a lack of assertion knowledge and test signature verification. This paper introduces A3Test, a novel DL-based approach to the generation of test cases, enhanced with assertion knowledge and a mechanism to verify consistency of the name and signatures of the tests. A3Test aims to adapt domain knowledge from assertion generation to test case generation. A3Test employs domain adaptation principles and introduces a verification approach to name consistency and test signatures. We evaluate its effectiveness using 5,278 focal methods from the Defects4j dataset. Our findings indicate that A3Test outperforms AthenaTest and ChatUniTest. A3Test generates 2.16% to 395.43% more correct test cases, achieves 2.17% to 34.29% higher method coverage, and 25.64% higher line coverage. A3Test achieves 2.13% to 12.20% higher branch coverage, 2.22% to 12.20% higher mutation scores, and 2.44% to 55.56% more correct assertions compared to both ChatUniTest and AthenaTest respectively for one iteration. When generating multiple test cases per method A3Test still shows improvements and comparable efficacy to ChatUnitTest. A survey of developers reveals that the majority of the participants 70.51% agree that test cases generated by A3Test are more readable than those generated by EvoSuite. A3Test significantly enhances test case generation through its incorporation of assertion knowledge and test signature verification, contributing to the generation of correct test cases.