On the training of recurrent neural networks

Abstract

This paper proposes a new approach which combines unsupervised and supervised learning for training recurrent neural networks (RNNs). In this approach, the weights between input and hidden layers were determined according to an unsupervised procedure relying on the Kohonen algorithm and the weights between hidden and output layers were updated according to a supervised procedure based on dynamic gradient descent method. The simulation results show that the proposed method performs well in comparison with the back propagation through time algorithm.