Hypothesis testing of diagnostic accuracy for multiple readers and multiple tests an anova approach with dependent observations

Abstract

A method is proposed for comparing the accuracy of diagnostic tests that rely on a reader's subjective interpretation of the results. An ANOVA approach is applied where the dependencies between the readers' estimates of diagnostic accuracy are handled by adjusting the usual F statistic for theestimated correlation. The distribution of the resulting test statistic is evaluated. The problem is particularly relevant to diagnostic radiology where multi-reader studies of the indices associated with the ROC curve serve important roles in evaluating the efficacy of diagnostic tests.