2017 · 16 citations · 45 references
ComputingComputer ScienceInsuranceArtificial IntelligenceMachine LearningNatural Language ProcessingAttention NetworksDeep LearningLanguage ModelsNatural Language UnderstandingQuestion AnsweringWord EmbeddingsAttentionAttention MechanismsNeural Language ModelsNeural Question AnsweringNeural Text RepresentationOpen-domain Question AnsweringQuestion Answering SystemsRecurrent Neural Networks
We propose a new attention mechanism for neural based question answering, which depends on varying granularities of the input. Previous work focused on augmenting recurrent neural networks for question answering systems with simple attention mechanisms which are a function of the similarity between a question embedding and an answer embeddings across time. We extend this by making the attention mechanism dependent on a global embedding of the answer attained using a separate network. We evaluate our system on InsuranceQA, a large question answering dataset. Our model outperforms current state-of-the-art results on InsuranceQA. Further, we examine which sections of text our attention mechanism focuses on, and explore its performance across different parameter settings.
45
Sepp Hochreiter, Jürgen Schmidhuber · Neural Computation · 1997 · 98.1K citations
Dropout: a simple way to prevent neural networks from overfitting
Nitish Srivastava, Geoffrey E. Hinton, Alex Krizhevsky et al. · 2014 · 34.3K citations
Distributed Representations of Words and Phrases and their Compositionality
Tomáš Mikolov, Ilya Sutskever, Kai Chen et al. · arXiv (Cornell University) · 2013 · 18.1K citations · Full text
Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
Jun‐Young Chung, Çağlar Gülçehre, Kyunghyun Cho et al. · arXiv (Cornell University) · 2014 · 10.8K citations · Full text