Forward-backward retraining of recurrent neural networks

Abstract

This paper describes the training of a recurrent neural network as the letter posterior probability estimator for a hidden Markov model, off-line handwriting recognition system. The network estimates posterior distributions for each of a series of frames representing sections of a handwritten word. The supervised training algorithm, backpropagation through time, requires target outputs to be provided for each frame. Three methods for deriving these targets are presented. A novel method based upon the forwardbackward algorithm is found to result in the recognizer with the lowest error rate. 1 Introduction In the field of off-line handwriting recognition, the goal is to read a handwritten document and produce a machine transcription. Such a system could be used for a variety of purposes, from cheque processing and postal sorting to personal correspondence reading for the blind or historical document reading. In a previous publication (Senior 1994) we have described a system based on a ...