Self organizing modular neural networks

Abstract

An argument is made for the advantages of self-scaling neural network learning algorithms as opposed to fixed-topology algorithms such as backpropagation. Cascade correlation is shown to be a self-scaling learning algorithm of great promise that suffers from some bad characteristics. These drawbacks include degradation of learning speed and quality with the size of the network and the development of deep networks with high fan-in rates to hidden units. The iterative atrophy algorithm is introduced as an enhancement of cascade correlation. It preserves the good features of cascade correlation while eliminating the worst characteristics. It is also shown that the new algorithm extends nicely into the realm of modular learning algorithms, where it is named modular iterative atrophy.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>