Incorporating Learnable Membrane Time Constant to Enhance Learning of Spiking Neural Networks

TLDR

Spiking Neural Networks are attractive for their temporal processing, low power, and biological plausibility, yet efficient learning remains difficult because current methods only tune weights and use identical, manually set membrane parameters that limit neuronal diversity. This study proposes a training algorithm that learns both synaptic weights and membrane time constants, inspired by regional variability in biological neurons. The algorithm jointly optimizes weights and per‑neuron membrane time constants and is evaluated on static datasets (MNIST, Fashion‑MNIST, CIFAR‑10) and neuromorphic datasets (N‑MNIST, CIFAR10‑DVS, DVS128 Gesture). Learning membrane time constants reduces sensitivity to initialization, accelerates training, and yields state‑of‑the‑art accuracy on almost all datasets with fewer time‑steps, while max‑pooling preserves information and is computationally efficient. Code is available at https://github.com/fangwei123456/Parametric-Leaky-Integrate-and-Fire-Spiking-Neuron.

Abstract

Spiking Neural Networks (SNNs) have attracted enormous research interest due to temporal information processing capability, low power consumption, and high biological plausibility. However, the formulation of efficient and high-performance learning algorithms for SNNs is still challenging. Most existing learning methods learn weights only, and require manual tuning of the membrane-related parameters that determine the dynamics of a single spiking neuron. These parameters are typically chosen to be the same for all neurons, which limits the diversity of neurons and thus the expressiveness of the resulting SNNs. In this paper, we take inspiration from the observation that membrane-related parameters are different across brain regions, and propose a training algorithm that is capable of learning not only the synaptic weights but also the membrane time constants of SNNs. We show that incorporating learnable membrane time constants can make the network less sensitive to initial values and can speed up learning. In addition, we reevaluate the pooling methods in SNNs and find that max-pooling will not lead to significant information loss and have the advantage of low computation cost and binary compatibility. We evaluate the proposed method for image classification tasks on both traditional static MNIST, Fashion-MNIST, CIFAR-10 datasets, and neuromorphic N-MNIST, CIFAR10-DVS, DVS128 Gesture datasets. The experiment results show that the proposed method outperforms the state-of-the-art accuracy on nearly all datasets, using fewer time-steps. Our codes are available at https://github.com/fangwei123456/Parametric-Leaky-Integrate-and-Fire-Spiking-Neuron.