arXiv (Cornell University) · 2015 · 3.3K citations · 21 references
ComputingComputer ScienceComputational LinguisticsNatural Language ProcessingText AnalyticsLanguage ModelingLanguage ModelsPart-of-speech TaggingSequence LabelingWord EmbeddingsWord Sense DisambiguationBidirectional Long Short-term Memory TaggingConditional Random FieldsLong Short-term Memory NetworksNeural Language ModelsSequence Representation LearningSequence Tagging
The paper proposes a range of LSTM‑based models for sequence tagging. The authors implement several architectures—LSTM, BI‑LSTM, LSTM‑CRF, and BI‑LSTM‑CRF—combining a bidirectional LSTM with a CRF layer to capture both past/future context and sentence‑level tag dependencies. The BI‑LSTM‑CRF achieves near state‑of‑the‑art accuracy on POS, chunking, and NER benchmarks, demonstrating robustness and reduced reliance on word embeddings.
In this paper, we propose a variety of Long Short-Term Memory (LSTM) based models for sequence tagging. These models include LSTM networks, bidirectional LSTM (BI-LSTM) networks, LSTM with a Conditional Random Field (CRF) layer (LSTM-CRF) and bidirectional LSTM with a CRF layer (BI-LSTM-CRF). Our work is the first to apply a bidirectional LSTM CRF (denoted as BI-LSTM-CRF) model to NLP benchmark sequence tagging data sets. We show that the BI-LSTM-CRF model can efficiently use both past and future input features thanks to a bidirectional LSTM component. It can also use sentence level tag information thanks to a CRF layer. The BI-LSTM-CRF model can produce state of the art (or close to) accuracy on POS, chunking and NER data sets. In addition, it is robust and has less dependence on word embedding as compared to previous observations.
21
Sepp Hochreiter, Jürgen Schmidhuber · Neural Computation · 1997 · 98.1K citations
Jeffrey L. Elman · Cognitive Science · 1990 · 10.8K citations
Speech recognition with deep recurrent neural networks
Alex Graves, Abdelrahman Mohamed, Geoffrey E. Hinton · 2013 · 8.8K citations