Support vector machines: A distance-based approach to multi-class classification

Abstract

One of the main tasks sought after with machine learning is classification. Support vector machines are one of the widely used machine learning algorithms for data classification. SVMs are by default binary classifiers, extending them to multi-class classifiers is a challenging on-going research problem. In this paper, we propose a new approach to constructing the multi-class classification function, where the structure and properties of the support vectors are exploited without altering the training procedure. Our contribution is based on the insight that one is not restricted to using the hyperplane-based decision function, resulting from the mathematical optimization problem. The proposed classification procedure considers the notion of distance between vectors in feature space. We show how, under the assumption of a normalized kernel, the distance between two vectors in feature space can be expressed solely in terms of their inner product. We apply both the original and proposed methods on synthetic datasets in a simulation setting, and then we argue that the proposed distance-based method represents a more rigorous and intuitive measure than the traditional hyperplane-based method.