Effect of CAD on radiologists' detection of lung nodules on thoracic CT scans: observer performance study

Abstract

The purpose of this study was to evaluate the effect of computer-aided diagnosis (CAD) on radiologists' performance for the detection of lung nodules on thoracic CT scans. Our computer system was designed using an independent training set of 94 CT scans in our laboratory. The data set for the observer performance study consisted of 48 CT scans. Twenty scans were collected from patient files at the University of Michigan, and 28 scans by the Lung Imaging Database Consortium (LIDC). All scans were read by multiple experienced thoracic radiologists to determine the true nodule locations, defined as any region identified by one or more expert radiologists as containing a nodule larger than 3 mm in diameter. Eighteen CT examinations were nodule-free, while the remaining 30 CT examinations contained a total of 73 nodules having a median size of 5.5 mm (range 3.0-36.4 mm). Four other study radiologists read the CT scans first without and then with CAD, and provided likelihood of nodule ratings for suspicious regions. Two of the study radiologists were fellowship trained in cardiothoracic radiology, and two were cardiothoracic radiology fellows. Freeresponse receiver-operating characteristic (FROC) curves were used to compare the two reading conditions. The computer system had a sensitivity of 79% (58/73) with an average of 4.9 marks per normal scan (88/18). Jackknife alternative FROC (JAFROC) analysis indicated that the improvement with CAD was statistically significant (p=0.03).