Surrogate Gradient Learning in Spiking Neural Networks: Bringing the Power of Gradient-Based Optimization to Spiking Neural Networks

TLDR

Spiking neural networks provide fault‑tolerant, energy‑efficient signal processing, and neuromorphic processors that emulate biological networks are emerging, creating an urgent need for methods that enable these systems to solve real‑world problems. The article aims to explain the challenges of training spiking neural networks and to guide readers through the key concepts of synaptic plasticity and data‑driven learning in the spiking setting. It reviews existing approaches and introduces surrogate‑gradient methods as a flexible and efficient solution to overcome these training challenges.

Abstract

Spiking neural networks (SNNs) are nature's versatile solution to fault-tolerant, energy-efficient signal processing. To translate these benefits into hardware, a growing number of neuromorphic spiking NN processors have attempted to emulate biological NNs. These developments have created an imminent need for methods and tools that enable such systems to solve real-world signal processing problems. Like conventional NNs, SNNs can be trained on real, domain-specific data; however, their training requires the overcoming of a number of challenges linked to their binary and dynamical nature. This article elucidates step-by-step the problems typically encountered when training SNNs and guides the reader through the key concepts of synaptic plasticity and data-driven learning in the spiking setting. Accordingly, it gives an overview of existing approaches and provides an introduction to surrogate gradient (SG) methods, specifically, as a particularly flexible and efficient method to overcome the aforementioned challenges.