Listen, attend and spell: A neural network for large vocabulary conversational speech recognition

TLDR

The authors introduce Listen, Attend and Spell (LAS), an end‑to‑end neural speech recognizer that transcribes utterances directly to characters without pronunciation models, HMMs, or other traditional components. LAS comprises a pyramidal recurrent encoder (listener) that processes filter‑bank spectra and an attention‑based recurrent decoder (speller) that emits each character conditioned on all previous characters and the entire acoustic sequence, thereby integrating acoustic, pronunciation, and language modeling into a single end‑to‑end system. On a Google voice search task, LAS achieves a 14.1 % WER without an external language model and 10.3 % with rescoring, outperforming the state‑of‑the‑art CLDNN‑HMM model’s 8.0 % WER.

Abstract

We present Listen, Attend and Spell (LAS), a neural speech recognizer that transcribes speech utterances directly to characters without pronunciation models, HMMs or other components of traditional speech recognizers. In LAS, the neural network architecture subsumes the acoustic, pronunciation and language models making it not only an end-to-end trained system but an end-to-end model. In contrast to DNN-HMM, CTC and most other models, LAS makes no independence assumptions about the probability distribution of the output character sequences given the acoustic sequence. Our system has two components: a listener and a speller. The listener is a pyramidal recurrent network encoder that accepts filter bank spectra as inputs. The speller is an attention-based recurrent network decoder that emits each character conditioned on all previous characters, and the entire acoustic sequence. On a Google voice search task, LAS achieves a WER of 14.1% without a dictionary or an external language model and 10.3% with language model rescoring over the top 32 beams. In comparison, the state-of-the-art CLDNN-HMM model achieves a WER of 8.0% on the same set.