2016 · 23 citations · 26 references
ComputingComputer EngineeringComputer ScienceMachine LearningMemoryNeural EngineeringMemory SystemsMemristive MaterialsNeural Circuit ModelingNeural CircuitsNeural NetworksFeedforward Neural NetworksMemristive Crossbar ArraysMemristive MemoryNeuromorphic HardwarePower EfficiencyRandom-access MemoryRecurrent Neural NetworksResistive Random-access Memory
We present a programmable high-efficient Recurrent Neural Network (RNN) with Synapses design using Resistive Random Access Memory (ReRAM). The presented ReRAM-RNN employs crossbar ReRAM arrays as synapses. A fast synapses programming methodology is realized by CMOS-based neuron with in-built programming circuitry. The simulations are performed using experimentally verified physical resistive switching model, instead of only functional models, providing better estimate of system speed and power efficiency. Simulation results show that ReRAM-RNN can provide higher computation efficiency and/or more compact design than software realization of RNN, and dedicated CMOS based digital-and analog-RNN. We show that the efficiency improvement of ReRAM-based neural network design is more significant in feedback networks than in feedforward networks.
26
Sepp Hochreiter, Jürgen Schmidhuber · Neural Computation · 1997 · 98.1K citations
Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
Jun‐Young Chung, Çağlar Gülçehre, Kyunghyun Cho et al. · arXiv (Cornell University) · 2014 · 10.8K citations · Full text
Speech recognition with deep recurrent neural networks
Alex Graves, Abdelrahman Mohamed, Geoffrey E. Hinton · 2013 · 8.8K citations
Recurrent neural network based language model
Tomáš Mikolov, Martin Karafiát, Lukáš Burget et al. · 2010 · 5.4K citations
Backpropagation through time: what it does and how to do it
Paul J. Werbos · Proceedings of the IEEE · 1990 · 4.9K citations