2020 · 32 citations · 9 references
ComputingComputer ScienceMachine LearningMemoryNatural Language ProcessingAttention NetworksNeural NetworksRepresentation LearningSequence LearningSequential RecommendationTime Series PredictionAttentionDeep Learning For Time Series ForecastingMachine Learning Forecasting ModelsMedication RecommendationMemory-augmented RecommendationRecurrent Neural Network RecommendationRecurrent Neural Networks
In recent years, sequential learning has been of great interest due to the advance of deep learning with applications in time-series forecasting, natural language processing, and speech recognition. Recurrent neural networks (RNNs) have achieved superior performance in single-view and synchronous multi-view sequential learning comparing to traditional machine learning models. However, the method remains less explored in asynchronous multi-view sequential learning, and the unalignment nature of multiple sequences poses a great challenge to learn the inter-view interactions. We develop an AMANet (Attention and Memory-Augmented Networks) architecture by integrating both attention and memory to solve asynchronous multi-view learning problem in general, and we focus on experiments in dual-view sequences in this paper. Self-attention and inter-attention are employed to capture intra-view interaction and inter-view interaction, respectively. History attention memory is designed to store the historical information of a specific object, which serves as local knowledge storage. Dynamic external memory is used to store global knowledge for each view. We evaluate our model in three tasks: medication recommendation from a patient's medical records, diagnosis-related group (DRG) classification from a hospital record, and invoice fraud detection through a company's taxation behaviors. The results demonstrate that our model outperforms all baselines and other state-of-the-art models in all tasks. Moreover, the ablation study of our model indicates that the inter-attention mechanism plays a key role in the model and it can boost the predictive power by effectively capturing the inter-view interactions from asynchronous views.
9
Sepp Hochreiter, Jürgen Schmidhuber · Neural Computation · 1997 · 98.1K citations
MIMIC-III, a freely accessible critical care database
Alistair E. W. Johnson, Tom Pollard, Lu Shen et al. · Scientific Data · 2016 · 8.1K citations · Full text