Enabling Spike-Based Backpropagation for Training Deep Neural Network Architectures

TLDR

Spiking Neural Networks are a promising computing paradigm, yet shallow architectures lack expressive power, converting trained ANNs fails to capture temporal dynamics, and directly training deep SNNs is hindered by the discontinuous, non‑differentiable spike generation. The authors aim to develop an approximate derivative method that incorporates the leaky integrate‑and‑fire neuron dynamics to enable spike‑based backpropagation. They implement this approximate derivative for LIF neurons, allowing direct training of deep convolutional SNNs with spike‑based backpropagation and analyze sparse event‑based inference to demonstrate computational efficiency. Experiments on VGG and Residual networks show that the proposed spike‑based learning achieves the highest classification accuracy on MNIST, SVHN, and CIFAR‑10 compared to other SNNs trained with spike‑based learning.

Abstract

Spiking Neural Networks (SNNs) have recently emerged as a prominent neural computing paradigm. However, the typical shallow SNN architectures have limited capacity for expressing complex representations, while training deep SNNs using input spikes has not been successful so far. Diverse methods have been proposed to get around this issue such as converting off-line trained deep Artificial Neural Networks (ANNs) to SNNs. However, the ANN-SNN conversion scheme fails to capture the temporal dynamics of a spiking system. On the other hand, it is still a difficult problem to directly train deep SNNs using input spike events due to the discontinuous, non-differentiable nature of the spike generation function. To overcome this problem, we propose an approximate derivative method that accounts for the leaky behavior of LIF neurons. This method enables training deep convolutional SNNs directly (with input spike events) using spike-based backpropagation. Our experiments show the effectiveness of the proposed spike-based learning strategy on deep networks (VGG and Residual architectures) by achieving the best classification accuracies in MNIST, SVHN and CIFAR-10 datasets compared to other SNNs trained with a spike-based learning. Moreover, we analyze sparse event-based computations to demonstrate the efficacy of the proposed SNN training method for inference operation in the spiking domain.