A method for controlling errors in two-class classification

Abstract

In data mining, two types of errors may occur when records are classified into two categories A and B: misclassifying an item in A as one of B and conversely misclassifying an item in B as one of A. Tight control over these two errors is needed in many practical situations since, for example, an error of one type may have far more serious consequences than one of the other type. Most previous work on two-class classifiers does not allow for such control. We describe a general approach that supports tight error control for two-class classification and that can utilize any two-class classification method as part of the decision mechanism. The main idea is to construct from the given training data a family of classifiers and then, using the training data once more, to estimate two distributions of certain vote totals. The error control is achieved via the two estimated distributions. The approach has been tested using several well-known classification problems. In each case, the two estimated distributions were very close to those obtained from verification data. Accordingly, good error control can be achieved.