An overview of some classical Growing Neural Networks and new developments

Abstract

The mapping capability of artificial neural networks (ANN) is dependent on their structure, i.e., the number of layers and the number of hidden units. There is no formal way of computing network topology as a function of the complexity of a problem. It is usually selected by trial-and-error and can be rather time consuming. Basically, we make use of two mechanisms that may modify the topology of the network: growth and pruning. This paper gives an overview of some classical Growing Neural Networks (GNN) and their new developments. This kind of GNN is also called the ANN with incremental learning. Firstly, some classical GNN with supervised learning are outlined which includes tiling algorithm, tower algorithm, upstart algorithm, cascade-correlation algorithm, restricted coulomb energy network and resource-allocation network. Secondly, a class of classical GNN with unsupervised learning is reviewed, such as self-organizing surfaces, evolve self-organizing maps, incremental grid growing and growing hierarchical self-organizing map. Thirdly, the new developments of GNN, including both supervised learning and unsupervised learning, are surveyed. The conclusion is given at the end of the paper.