Bidirectional recurrent neural networks

TLDR

The BRNN can be trained without the limitation of using input information just up to a preset future frame. The paper extends a regular RNN to a bidirectional RNN and demonstrates how this structure can be modified to efficiently estimate the conditional posterior probability of complete symbol sequences without assuming a distribution shape. The BRNN is obtained by extending a regular RNN and training it simultaneously in positive and negative time directions, with its structure and training procedure detailed. Regression and classification experiments on artificial data and phoneme classification on the TIMIT database show that the proposed BRNN outperforms other approaches, and additional real‑data experiments confirm this advantage.

Abstract

In the first part of this paper, a regular recurrent neural network (RNN) is extended to a bidirectional recurrent neural network (BRNN). The BRNN can be trained without the limitation of using input information just up to a preset future frame. This is accomplished by training it simultaneously in positive and negative time direction. Structure and training procedure of the proposed network are explained. In regression and classification experiments on artificial data, the proposed structure gives better results than other approaches. For real data, classification experiments for phonemes from the TIMIT database show the same tendency. In the second part of this paper, it is shown how the proposed bidirectional structure can be easily modified to allow efficient estimation of the conditional posterior probability of complete symbol sequences without making any explicit assumption about the shape of the distribution. For this part, experiments on real data are reported.