Exploiting the Cost (In)sensitivity of Decision Tree Splitting Criteria

TLDR

There are two opposite ways of dealing with class imbalance. The paper investigates how decision‑tree splitting criteria and pruning methods are affected by misclassification costs or class‑distribution changes. The authors compare two approaches: combining a cost‑insensitive splitting criterion with a cost‑insensitive pruning method, and growing a cost‑independent tree followed by cost‑sensitive pruning. Cost‑insensitive splitting criteria perform as well as or better than cost‑sensitive criteria in terms of expected misclassification cost.

Abstract

This paper investigates how the splitting criteria and pruning methods of decision tree learning algorithms are influenced by misclassification costs or changes to the class distribution. Splitting criteria that are relatively insensitive to costs (class distributions) are found to perform as well as or better than, in terms of expected misclassification cost, splitting criteria that are cost sensitive. Consequently there are two opposite ways of dealing with imbalance. One is to combine a costinsensitive splitting criterion with a cost insensitive pruning method to produce a decision tree algorithm little affected by cost or prior class distribution. The other is to grow a cost-independent tree which is then pruned in a cost-sensitive manner.