A Generalized Mixture Framework for Multi-label Classification

Abstract

We develop a novel probabilistic ensemble framework for multi-label classification that is based on the <i>mixtures-of-experts</i> architecture. In this framework, we combine multi-label classification models in the <i>classifier chains family</i> that decompose the class posterior distribution <i>P</i>(<i>Y</i>₁, …, <i>Y_d</i> |<b>X</b>) using a product of posterior distributions over components of the output space. Our approach captures different input-output and output-output relations that tend to change across data. As a result, we can recover a rich set of dependency relations among inputs and outputs that a single multi-label classification model cannot capture due to its modeling simplifications. We develop and present algorithms for learning the mixtures-of-experts models from data and for performing multi-label predictions on unseen data instances. Experiments on multiple benchmark datasets demonstrate that our approach achieves highly competitive results and outperforms the existing state-of-the-art multi-label classification methods.