What's under the ROC? An Introduction to Receiver Operating Characteristics Curves

TLDR

Dichotomizing continuous scales into normal and abnormal groups often yields overlapping score distributions, producing false positives and false negatives that cannot be eliminated by simply changing the cut point. The paper explains how to calculate and compare ROC curves and discusses factors that influence the choice of an optimal cut point. ROC curves evaluate a test's discriminative power, aid in selecting the best threshold, and allow comparison of multiple tests.

Abstract

It is often necessary to dichotomize a continuous scale to separate respondents into normal and abnormal groups. However, because the distributions of the scores in these 2 groups most often overlap, any cut point that is chosen will result in 2 types of errors: false negatives (that is, abnormal cases judged to be normal) and false positives (that is, normal cases placed in the abnormal group). Changing the cut point will alter the numbers of erroneous judgments but will not eliminate the problem. A technique called receiver operating characteristic (ROC) curves allows us to determine the ability of a test to discriminate between groups, to choose the optimal cut point, and to compare the performance of 2 or more tests. We discuss how to calculate and compare ROC curves and the factors that must be considered in choosing an optimal cut point.