Tlie Measurement of Performance in Probabilistic Diagnosis

Abstract

Within the framework of diagnostic probability prediction the problem of measuring discriminatory ability is operationally defined as the problem of measuring the agreement between probabilistic predictions and actual outcomes. We present a number of so-called scoring rules developed to this end. Most of these are continuous functions of the assigned probabilities. Discontinuous rules, including conventional non-error rates, are discussed by way of contrast. The concept of properness of a scoring-rule is discussed and the desirability of properness is argued. Separate sections deal with the problems connected with uncommon diseases and methods utilizing subdivisions of the patient material. The distinction between the three concepts of discriminatory ability, sharpness and reliability is explained. The evaluation tools developed are applied to previously presented data from the Copenhagen Acute Abdominal Pain Study.