On improving recurrent neural network for image classification

Abstract

The paper proposes a new method for improving the performance of Recurrent Neural Networks. The proposed method uses two parallel recurrent layers which execute independent of each other. The final output of recurrent layer at any time step is computed as the mean of the modulus of the output of these two layers. The proposed method attempts to overcome the limitations of the existing Recurrent Neural Networks with regard to the flow of gradient. Comparative performance of the proposed method has been carried out with Long Short Term Memory (LSTM) and Identity initialized RNN (IRNN), the latest improved version of RNN for classification of images. On benchmark image datasets, it has been shown that the proposed method outperforms both IRNN and LSTM.