A Reservoir-based Convolutional Spiking Neural Network for Gesture Recognition from DVS Input

Abstract

Mammalian neural circuits respond to different sensory stimuli by firing spikes at particular times. Closely mimicking this phenomenon, the evolving 3rd generation neural networks, known as Spiking Neural Networks (SNNs), are found to be capable of memorizing and learning from the spatio-temporal spike patterns. This makes SNN applicable in identification of human actions and gestures, especially in the robotics domain. The paradigm is also suited for Neuromorphic Systems leading to less energy intensive applications. In this work, we present a novel spiking neural network constituting multiple convolutional layers and a reservoir layer to extract spatial and temporal features respectively from human gesture videos captured with DVS camera. We achieved more than 95% Top-3 accuracy on IBM DVS dataset and we claim that the performance of our network is better in terms of accuracy vs. learning parameters ratio when compared to other networks.