Using diversity in cluster ensembles

Abstract

The pairwise approach to cluster ensembles uses multiple partitions, each of which constructs a coincidence matrix between all pairs of objects. The matrices for the partitions are then combined and a final clustering is derived thereof. Here we study the diversity within such cluster ensembles. Based on this, we propose a variant of the generic ensemble method where the number of overproduced clusters is chosen randomly for every ensemble member (partition). Using three artificial sets we show that this approach increases the spread of the diversity within the ensemble thereby leading to a better match with the known cluster labels. Experimental results with three real data sets are also reported.