A Resource-Allocating Network for Function Interpolation

TLDR

The network dynamically allocates new computational units for novel input patterns, adjusts existing unit parameters via LMS gradient descent when performance is adequate, and adds units when errors occur, allowing learning without repeating patterns and with local input responsiveness. The resource‑allocating network achieves compact representations, learns rapidly, and outperforms backpropagation networks on the Mackey‑Glass chaotic time series while using a comparable number of synapses.

Abstract

We have created a network that allocates a new computational unit whenever an unusual pattern is presented to the network. This network forms compact representations, yet learns easily and rapidly. The network can be used at any time in the learning process and the learning patterns do not have to be repeated. The units in this network respond to only a local region of the space of input values. The network learns by allocating new units and adjusting the parameters of existing units. If the network performs poorly on a presented pattern, then a new unit is allocated that corrects the response to the presented pattern. If the network performs well on a presented pattern, then the network parameters are updated using standard LMS gradient descent. We have obtained good results with our resource-allocating network (RAN). For predicting the Mackey-Glass chaotic time series, RAN learns much faster than do those using backpropagation networks and uses a comparable number of synapses.