Evaluating the realism of synthetically generated mammographic lesions: an observer study

Abstract

A method has been developed for generating synthetic masses that exhibit the appearance of real breast cancers in mammograms. To be clinically useful, the synthetic masses must appear sufficiently realistic, even to expert mammogram readers. This paper presents the results of an observer study in which 10 expert mammogram readers at the Nightingale Centre, Manchester attempted to distinguish between real and synthetically generated masses. Each reader rated a set of 30 real and 30 synthetics masses on a scale ranging from "definitely real" to "definitely synthetic". ROC curves were fitted to their responses and the area-under-curve (AUC) used to quantify the ability of a reader to identify synthetic masses. The mean AUC was 0.70±0.09, showing the readers were able to identify synthetic masses at a rate statistically better than chance and suggesting that further improvements must be made to the mass synthesis method. Analysis of individual AUC scores showed reader performance was not affected by job type (radiologist versus breast physician/radiographer) or experience.