An Alternative Model for Mixtures of Experts

Abstract

An alternative model is proposed for mixtures-of-experts, by utilizing a different parametric form for the gating network. The modified model is trained by an EM algorithm. In comparison with earlier models---trained by either EM or gradient ascent---there is no need to select a learning stepsize to guarantee the convergence of the learning procedure. We report simulation experiments which show that the new architecture yields significantly faster convergence. We also apply the new model to two problems domains: piecewise nonlinear function approximation and combining multiple previously trained classifiers.