A digital hardware pulse-mode neuron with piecewise linear activation function

Abstract

This paper proposes a new type of digital pulse-mode neuron that employs piecewise-linear function as its activation function. The neuron is implemented on field programmable gate array (FPGA) and tested by experiments. As well as theoretical analysis, the experimental results show that the piecewise-linear function of the proposed neuron is programmable and robust against the change in the number of input signals. To demonstrate the effect of piecewise-linear activation function, pulse-mode multilayer neural network with on-chip learning is implemented on FPGA with the proposed neuron, and its learning performance is verified by experiments. By approximating the sigmoid function by the piecewise-linear function, the convergence rate of the learning and generalization capability are improved.