Smooth Hinging Hyperplanes - An Alternative to Neural Nets

Abstract

Recently a novel approach to nonlinear function approximation using hinging hyperplanes, was reported by L. Breiman [3]. In this contribution we have combined smooth hinging hyperplanes and the efficient initialization procedure existing for hinging hyperplanes in [3], with a Gauss-Newton procedure, see [4], to perform the final adjustment of the smooth hinging hyperplanes. This combination uses the property of the hinge functions that makes them effective, namely that there is a simple and computationally efficient method for locating hinges. The result of the hinge finding procedure is then used as an initial value to the Gauss-Newton procedure applied on the smoothed hinges. The smooth hinging hyperplanes and neural networks are related, but the significant problem of choosing initial parameters of neural networks, in this case, is circumvented. Further, the influence of the choice of initial value of the "smoothness parameter" on the final approximating function estimate, is investigated. A recommendation on how to choose an initial value is given.