Classification using discriminative restricted Boltzmann machines

Abstract

Recently, many applications for Restricted Boltzmann Machines (RBMs) have been developed for a large variety of learning problems.However, RBMs are usually used as feature extractors for another learning algorithm or to provide a good initialization for deep feed-forward neural network classifiers, and are not considered as a standalone solution to classification problems.In this paper, we argue that RBMs provide a self-contained framework for deriving competitive non-linear classifiers.We present an evaluation of different learning algorithms for RBMs which aim at introducing a discriminative component to RBM training and improve their performance as classifiers.This approach is simple in that RBMs are used directly to build a classifier, rather than as a stepping stone.Finally, we demonstrate how discriminative RBMs can also be successfully employed in a semi-supervised setting.