Complexity measures of supervised classification problems

TLDR

The authors propose using the complexity measurement space to describe a classifier’s domain of competence. They studied geometrical complexity measures of class boundaries and suggest employing the resulting measurement space to characterize a classifier’s domain of competence. Real‑world classification problems exhibit distinct geometrical complexity structures compared to random labelings, revealing at least two independent difficulty factors, and these insights can guide static and dynamic classifier selection for specific problems and their subproblems.

Abstract

We studied a number of measures that characterize the difficulty of a classification problem, focusing on the geometrical complexity of the class boundary. We compared a set of real-world problems to random labelings of points and found that real problems contain structures in this measurement space that are significantly different from the random sets. Distributions of problems in this space show that there exist at least two independent factors affecting a problem's difficulty. We suggest using this space to describe a classifier's domain of competence. This can guide static and dynamic selection of classifiers for specific problems as well as subproblems formed by confinement, projection, and transformations of the feature vectors.