Optimum block-adaptive learning algorithm for error back-propagation networks

Abstract

An optimum block-adaptive learning rate (OBALR) backpropagation (BP) algorithm for training feedforward neural networks with an arbitrary number of neuron layers is described. The algorithm uses block-smoothed gradient as direction for descent and no momentum term, but produces an optimum block-adaptive learning rate which is constant within each block and is updated adaptively at the beginning of each block iteration so that it is kept optimum in a sense of minimizing the approximate output mean-square error of the block. Several computer simulations were tested on learning a deterministic chaos time-series mapping. The OBALR BP algorithm not only overcame the difficulty in choosing good values of the two parameters, but also provided significant improvement on learning speed and descent capability over the standard BP algorithm.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>