Improving the learning speed of 2-layer neural networks by choosing initial values of the adaptive weights

Abstract

The authors describe how a two-layer neural network can approximate any nonlinear function by forming a union of piecewise linear segments. A method is given for picking initial weights for the network to decrease training time. The authors have used the method to initialize adaptive weights over a large number of different training problems and have achieved major improvements in learning speed in every case. The improvement is best when a large number of hidden units is used with a complicated desired response. The authors have used the method to train the truck-backer-upper and were able to decrease the training time from about two days to four hours