Training of radial basis function classifiers with resilient propagation and variational Bayesian inference

Abstract

For classification tasks, the application of generative classifiers sometimes has advantages over the use of exclusively discriminative classifiers because loss functions can be considered or rejection criteria can be defined more easily, for instance. We show how a radial basis function (RBF) network with multivariate (elliptical) Gaussian basis functions can be trained in two different ways to obtain a classifier with either a more generative or a more discriminative behavior. Our generative classifier allows a probabilistic interpretation of the external outputs (posterior probability of class membership) and the hidden neurons' activations (posterior probability of a component of the model). For that purpose a variational Bayesian inference approach is applied, which also finds an appropriate number of hidden neurons (i.e., components) ldquoon the flyrdquo. A discriminative classifier is obtained using the resilient propagation training technique. We investigate the properties of the two training techniques in detail by introducing a measure for generative properties of the trained classifiers and by comparing these classifiers on various data sets.