Using the receiver operating characteristic to asses the performance of neural classifiers

Abstract

As artificial neural networks continue to find usefulness in fields which historically favor more traditional statistical methods, the neural practitioner inevitably learns of useful techniques well known to statisticians which have yet to find widespread use in the field of neural networks. One such method, commonly used in medical screening and diagnosis, is receiver operating characteristic (ROC) analysis. ROC analysis is easily applied to a neural classifier, yet today is rarely used to assess the performance of neural classifiers outside of the medical and signal detection fields. We show how ROC analysis can be applied to neural network classifiers and demonstrate its usefulness by applying it to the diagnosis of psychiatric illness. Benefits of ROC analysis include a more robust description of the network's predictive ability and a convenient way to "tune" an already trained network according to differential costs of misclassification and varying prior probabilities of class occurrences.