Hybrid speech recognition with Deep Bidirectional LSTM

TLDR

Deep Bidirectional LSTM recurrent neural networks have achieved state‑of‑the‑art performance on the TIMIT speech database, yet their RNN‑specific objective functions hinder integration with large‑vocabulary speech recognition systems. This study evaluates DBLSTM as an acoustic model within a conventional neural‑network HMM hybrid system and seeks to identify strategies for translating its superior frame‑level accuracy into lower word error rates. The DBLSTM‑HMM hybrid attains TIMIT results comparable to prior work, surpasses GMM and deep‑network baselines on a subset of the Wall Street Journal corpus, but yields only modest word‑error‑rate gains despite markedly higher frame‑level accuracy, indicating its strength lies in tasks dominated by acoustic modeling.

Abstract

Deep Bidirectional LSTM (DBLSTM) recurrent neural networks have recently been shown to give state-of-the-art performance on the TIMIT speech database. However, the results in that work relied on recurrent-neural-network-specific objective functions, which are difficult to integrate with existing large vocabulary speech recognition systems. This paper investigates the use of DBLSTM as an acoustic model in a standard neural network-HMM hybrid system. We find that a DBLSTM-HMM hybrid gives equally good results on TIMIT as the previous work. It also outperforms both GMM and deep network benchmarks on a subset of the Wall Street Journal corpus. However the improvement in word error rate over the deep network is modest, despite a great increase in framelevel accuracy. We conclude that the hybrid approach with DBLSTM appears to be well suited for tasks where acoustic modelling predominates. Further investigation needs to be conducted to understand how to better leverage the improvements in frame-level accuracy towards better word error rates.