Going Deeper in Spiking Neural Networks: VGG and Residual Architectures

Abhronil Sengupta, Yuting Ye, Robert Wang, Chiao Liu, Kaushik Roy

Frontiers in Neuroscience · 2019 · 59 citations · 23 references

DOIFull text

Open access

Concepts

TL;DR

Spiking Neural Networks have gained traction as a low‑power, event‑driven neuromorphic paradigm, yet their use in machine learning has largely been confined to shallow models for simple tasks. This work introduces a new algorithmic method for constructing deep SNNs and evaluates it on challenging visual recognition benchmarks such as CIFAR‑10 and ImageNet. The method is applied to VGG and Residual network architectures, and analysis of the resulting sparse, event‑driven activity demonstrates reduced hardware overhead. The resulting deep SNNs achieve markedly higher accuracy than the current state‑of‑the‑art approaches.

Abstract

Over the past few years, Spiking Neural Networks (SNNs) have become popular as a possible pathway to enable low-power event-driven neuromorphic hardware. However, their application in machine learning have largely been limited to very shallow neural network architectures for simple problems. In this paper, we propose a novel algorithmic technique for generating an SNN with a deep architecture, and demonstrate its effectiveness on complex visual recognition problems such as CIFAR-10 and ImageNet. Our technique applies to both VGG and Residual network architectures, with significantly better accuracy than the state-of-the-art. Finally, we present analysis of the sparse event-driven computations to demonstrate reduced hardware overhead when operating in the spiking domain.

References

23