Micromachines · 2021 · 22 citations · 16 references
EngineeringComputer EngineeringComputer ScienceComputer ArchitectureLow-power DesignMicroelectronicsIntegrated Circuit DesignNeural CircuitsNeural NetworksActivation FunctionsConvolutional Neural NetworksEnergy-delay TradeoffEnergy-efficient ComputationMultiply-accumulate UnitsOverhead ReductionPower ConsumptionVon Neumann Architecture
Computing-In-Memory (CIM), based on non-von Neumann architecture, has lately received significant attention due to its lower overhead in delay and higher energy efficiency in convolutional and fully-connected neural network computing. Growing works have given the priority to researching the array of memory and peripheral circuits to achieve multiply-and-accumulate (MAC) operation, but not enough attention has been paid to the high-precision hardware implementation of non-linear layers up to now, which still causes time overhead and power consumption. Sigmoid is a widely used non-linear activation function and most of its studies provided an approximation of the function expression rather than totally matched, inevitably leading to considerable error. To address this issue, we propose a high-precision circuit implementation of the sigmoid, matching the expression exactly for the first time. The simulation results with the SMIC 40 nm process suggest that the proposed circuit implemented high-precision sigmoid perfectly achieves the properties of the ideal sigmoid, showing the maximum error and average error between the proposed simulated sigmoid and ideal sigmoid is 2.74% and 0.21%, respectively. In addition, a multi-layer convolutional neural network based on CIM architecture employing the simulated high-precision sigmoid activation function verifies the similar recognition accuracy on the test database of handwritten digits compared to utilize the ideal sigmoid in software, with online training achieving 97.06% and with offline training achieving 97.74%.
16
Gradient-based learning applied to document recognition
Yann LeCun, Léon Bottou, Yoshua Bengio et al. · Proceedings of the IEEE · 1998 · 58.2K citations · Full text
Backpropagation Applied to Handwritten Zip Code Recognition
Yann LeCun, Bernhard E. Boser, J. S. Denker et al. · Neural Computation · 1989 · 11.9K citations
Fully hardware-implemented memristor convolutional neural network
Peng Yao, Huaqiang Wu, Bin Gao et al. · Nature · 2020 · 2.1K citations
Engineering, Computer Engineering, Electrical Engineering +12