arXiv (Cornell University) · 2018 · 116 citations · 0 references
ComputingComputer SystemsComputational NeuroscienceNeural EngineeringSystems NeuroscienceDeep LearningNeural Circuit DynamicsNeural NetworksNeuronal DynamicsSpiking Neural NetworksBackpropagationConvolutional Neural NetworksConvolutional Spiking Neural NetworksDeep Spiking Neural NetworksFully Convolutional NetworksMismatchNeuromorphic HardwareSpike Train Dynamics
Spiking neural networks (SNNs) are positioned to enable spatio-temporal\ninformation processing and ultra-low power event-driven neuromorphic hardware.\nHowever, SNNs are yet to reach the same performances of conventional deep\nartificial neural networks (ANNs), a long-standing challenge due to complex\ndynamics and non-differentiable spike events encountered in training. The\nexisting SNN error backpropagation (BP) methods are limited in terms of\nscalability, lack of proper handling of spiking discontinuities, and/or\nmismatch between the rate-coded loss function and computed gradient. We present\na hybrid macro/micro level backpropagation (HM2-BP) algorithm for training\nmulti-layer SNNs. The temporal effects are precisely captured by the proposed\nspike-train level post-synaptic potential (S-PSP) at the microscopic level. The\nrate-coded errors are defined at the macroscopic level, computed and\nback-propagated across both macroscopic and microscopic levels. Different from\nexisting BP methods, HM2-BP directly computes the gradient of the rate-coded\nloss function w.r.t tunable parameters. We evaluate the proposed HM2-BP\nalgorithm by training deep fully connected and convolutional SNNs based on the\nstatic MNIST [14] and dynamic neuromorphic N-MNIST [26]. HM2-BP achieves an\naccuracy level of 99.49% and 98.88% for MNIST and N-MNIST, respectively,\noutperforming the best reported performances obtained from the existing SNN BP\nalgorithms. Furthermore, the HM2-BP produces the highest accuracies based on\nSNNs for the EMNIST [3] dataset, and leads to high recognition accuracy for the\n16-speaker spoken English letters of TI46 Corpus [16], a challenging\npatio-temporal speech recognition benchmark for which no prior success based on\nSNNs was reported. It also achieves competitive performances surpassing those\nof conventional deep learning models when dealing with asynchronous spiking\nstreams.\n