LLMs for Intelligent Software Testing: A Comparative Study

Abstract

The need for effective and timely testing processes has become critical in the constantly changing field of software development. Large Language Models (LLMs) have demonstrated promise in automating test case creation, defect detection, and other software testing tasks through the use of the capabilities of machine/deep learning and natural language processing. This work explores the field of intelligent software testing, with a focus on the use of LLMs in this context. The purpose of this comparative study is to assess the corpus of research in the field in terms of used LLMs, how to interact with them, the use of fine-tuning, and prompt engineering, and explore the different technologies and testing types automated using LLMs. The findings of this study not only contribute to the growing body of knowledge on intelligent software testing but also guide fellow researchers and industry engineers in selecting the most suitable LLM for their specific testing needs.