Classification assessment methods

TLDR

Classification techniques are widely used across scientific fields, and evaluating them involves various scalar and graphical metrics that must be interpreted correctly. The paper aims to provide a comprehensive overview of classification assessment measures for researchers. The authors present an overview that defines confusion matrices, explains numerous classification metrics, discusses their behavior with balanced and imbalanced data, and illustrates calculations and graphical representations such as ROC, PR, and DET curves through step‑by‑step examples. The paper details how each metric behaves under balanced versus imbalanced data conditions.

Abstract

Abstract Classification techniques have been applied to many applications in various fields of sciences. There are several ways of evaluating classification algorithms. The analysis of such metrics and its significance must be interpreted correctly for evaluating different learning algorithms. Most of these measures are scalar metrics and some of them are graphical methods. This paper introduces a detailed overview of the classification assessment measures with the aim of providing the basics of these measures and to show how it works to serve as a comprehensive source for researchers who are interested in this field. This overview starts by highlighting the definition of the confusion matrix in binary and multi-class classification problems. Many classification measures are also explained in details, and the influence of balanced and imbalanced data on each metric is presented. An illustrative example is introduced to show (1) how to calculate these measures in binary and multi-class classification problems, and (2) the robustness of some measures against balanced and imbalanced data. Moreover, some graphical measures such as Receiver operating characteristics (ROC), Precision-Recall, and Detection error trade-off (DET) curves are presented with details. Additionally, in a step-by-step approach, different numerical examples are demonstrated to explain the preprocessing steps of plotting ROC, PR, and DET curves.