Exploiting Generative Models in Discriminative Classifiers

TLDR

Generative models such as HMMs handle missing data and variable‑length sequences, while discriminative methods like SVMs build flexible decision boundaries and often achieve superior classification performance. An ideal classifier should combine these two complementary approaches. The authors derive kernel functions from generative probability models to use in discriminative SVMs, enabling the combination. The approach is theoretically justified and yields substantial performance gains in DNA and protein sequence classification.

Abstract

Generative probability models such as hidden Markov models provide a principled way of treating missing information and dealing with variable length sequences. On the other hand, discriminative methods such as support vector machines enable us to construct flexible decision boundaries and often result in classification performance superior to that of the model based approaches. An ideal classifier should combine these two complementary approaches. In this paper, we develop a natural way of achieving this combination by deriving kernel functions for use in discriminative methods such as support vector machines from generative probability models. We provide a theoretical justification for this combination as well as demonstrate a substantial improvement in the classification performance in the context of DNA and protein sequence analysis.