Empirical evaluation of the tarantula automatic fault-localization technique

TLDR

Locating faults in programs is costly, prompting automated fault‑localization techniques that rank statements by suspiciousness, and prior studies have compared four such techniques on subject programs. This study compares the Tarantula technique with those four previously evaluated techniques to determine its relative effectiveness. The authors conducted an experiment that overviews Tarantula and the four other techniques, describes the experimental setup, and reports the results. Tarantula consistently outperforms the other four techniques in fault‑localization effectiveness while matching the least expensive technique in efficiency.

Abstract

The high cost of locating faults in programs has motivated the development of techniques that assist in fault localization by automating part of the process of searching for faults. Empirical studies that compare these techniques have reported the relative effectiveness of four existing techniques on a set of subjects. These studies compare the rankings that the techniques compute for statements in the subject programs and the effectiveness of these rankings in locating the faults. However, it is unknown how these four techniques compare with Tarantula, another existing fault-localization technique, although this technique also provides a way to rank statements in terms of their suspiciousness. Thus, we performed a study to compare the Tarantula technique with the four techniques previously compared. This paper presents our study---it overviews the Tarantula technique along with the four other techniques studied, describes our experiment, and reports and discusses the results. Our studies show that, on the same set of subjects, the Tarantula technique consistently outperforms the other four techniques in terms of effectiveness in fault localization, and is comparable in efficiency to the least expensive of the other four techniques.