Neuromorphic Computing and Engineering · 2025 · 14 citations · 40 references
ComputingComputer ScienceComputational NeuroscienceMachine LearningNeural EngineeringDeep LearningHardware AccelerationNeural NetworksSpeech RecognitionSpiking Neural NetworksEnergy-efficient AlgorithmsEnergy-efficient ComputationGradient DescentNeuromorphic DevicesNeuromorphic HardwareRecurrent Spiking Neural NetworksSpiking Neural Network ModelsSurrogate Gradient Learning
Abstract Event-based machine learning promises more energy-efficient AI on future neuromorphic hardware. Here, we investigate how the recently discovered Eventprop algorithm for gradient descent on exact gradients in spiking neural networks (SNNs) can be scaled up to challenging keyword recognition benchmarks. We implemented Eventprop in the GPU-enhanced neural networks framework (GeNN) and used it for training recurrent SNNs on the Spiking Heidelberg Digits (SHD) and Spiking Speech Commands (SSC) datasets. We found that learning depended strongly on the loss function and extended Eventprop to a wider class of loss functions to enable effective training. We then tested a large number of data augmentations and regularisations as well as exploring different network structures; and heterogeneous and trainable timescales. We found that when combined with two specific augmentations, the right regularisation and a delay line input, Eventprop networks with one recurrent layer achieved state-of-the-art performance on SHD and good accuracy on SSC. In comparison to a leading surrogate-gradient-based SNN training method, our GeNN Eventprop implementation is 3× faster and uses 4× less memory. This work is a significant step towards a low-power neuromorphic alternative to current machine learning paradigms.
40
Loihi: A Neuromorphic Manycore Processor with On-Chip Learning
Mike Davies, Narayan Srinivasa, Tsung-Han Lin et al. · IEEE Micro · 2018 · 3.7K citations
Engineering, Computer Engineering, Computational Neuroscience +16
Spatio-Temporal Backpropagation for Training High-Performance Spiking Neural Networks
Yujie Wu, Lei Deng, Guoqi Li et al. · Frontiers in Neuroscience · 2018 · 1.1K citations · Full text
A solution to the learning dilemma for recurrent networks of spiking neurons
Guillaume Bellec, Franz Scherr, Anand Subramoney et al. · Nature Communications · 2020 · 493 citations · Full text