Hybrid Decision Tree Learners with Alternative Leaf Classifiers: An Empirical Study

Abstract

. There has been surprisingly little research so far that systematically investigated the possibility of constructing hybrid learning algorithms by simple local modications to decision tree learners. In this paper we analyze three variants of a C4.5-style learner, introducing alternative leaf models (Naive Bayes, IB1, and multi-response linear regression, respectively) which can replace the original C4.5 leaf nodes during reduced error post-pruning. We empirically show that these simple modi cations can improve upon the performance of the original decision tree algorithm and even upon both constituent algorithms. We see this as a step towards the construction of learners that locally optimize their bias for dierent regions of the instance space. 1 Introduction Tree-based learning methods are widely used for machine learning and data mining applications. These methods have a long tradition and are commonly known since the works of Breiman et al. [3] and Quinlan [16]. The...