Structured multicategory support vector machines with analysis of variance decomposition

Abstract

The support vector machine has been a popular choice of classification method for many applications in machine learning. While it often outperforms other methods in terms of classification accuracy, the implicit nature of its solution renders the support vector machine less attractive in providing insights into the relationship between covariates and classes. Use of structured kernels can remedy the drawback. Borrowing the flexible model-building idea of functional analysis of variance decomposition, we consider multicategory support vector machines with analysis of variance kernels in this paper. An additional penalty is imposed on the sum of weights of functional subspaces, which encourages a sparse representation of the solution. Incorporation of the additional penalty enhances the interpretability of a resulting classifier with often improved accuracy. The proposed method is demonstrated through simulation studies and an application to real data.