Artificial Intelligence Science and Engineering · 2025 · 12 citations · 49 references
ComputingComputer ScienceComputer SystemsArtificial IntelligenceMachine LearningMemoryDeep LearningHardware AccelerationMemory SystemsNeural EncodingNeural NetworksScalable Machine LearningEnergy-efficient ComputationLarge Language ModelsNeural Representation LearningNeuromorphic DevicesNeuromorphic HardwareScalabilityTransferable Representation LearningTransformer Models
The rapid advancement of deep learning and the emergence of large-scale neural models, such as bidirectional encoder representations from transformers (BERT), generative pre-trained transformer (GPT), and large language model Meta AI (LLaMa), have brought significant computational and energy challenges. Neuromorphic computing presents a biologically inspired approach to addressing these issues, leveraging event-driven processing and in-memory computation for enhanced energy efficiency. This survey explores the intersection of neuromorphic computing and large-scale deep learning models, focusing on neuromorphic models, learning methods, and hardware. We highlight transferable techniques from deep learning to neuromorphic computing and examine the memory-related scalability limitations of current neuromorphic systems. Furthermore, we identify potential directions to enable neuromorphic systems to meet the growing demands of modern AI workloads.
49
Deep Residual Learning for Image Recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren et al. · 2016 · 222.1K citations · Full text
Sepp Hochreiter, Jürgen Schmidhuber · Neural Computation · 1997 · 98.1K citations
Dmitri B. Strukov, Gregory S. Snider, Duncan R. Stewart et al. · Nature · 2008 · 11.5K citations
Engineering, Electrical Engineering, Electronic Materials +5
Memristor-The missing circuit element
Leon O. Chua · IEEE Transactions on Circuit Theory · 1971 · 10K citations
Loihi: A Neuromorphic Manycore Processor with On-Chip Learning
Mike Davies, Narayan Srinivasa, Tsung-Han Lin et al. · IEEE Micro · 2018 · 3.7K citations
Engineering, Computer Engineering, Computational Neuroscience +16