IEEE Transactions on Industrial Electronics · 2023 · 17 citations · 27 references
EngineeringComputer EngineeringElectrical EngineeringAnalog ElectronicsComputer ArchitectureMicroelectronicsAnalog Integrated CircuitsMemory DevicesNeural NetworksActivationActivation FunctionsAnalog Circuit DesignIndustrial ElectronicsMemristive CircuitsNeuromorphic DevicesPhase-change MemoryPower Efficiency
Artificial neural networks (ANNs) have recently made a significant impact on the field of industrial electronics. Increasing efforts have been focused on implementing ANN using more efficient and compact architectures to achieve improved computational performance and power efficiency. The activation function of the nonlinear layers is crucial for the successful implementation of ANN, as it enables the network to possess generalization ability. In this study, we implemented the hyperbolic tangent function (tanh) using an analog circuit based on phase-change memory (PCM), which is a mature nonvolatile technology. This fully analog PCM-based tanh function is naturally compatible with analog-memory-based ANN architectures, such as the memristive neural networks (MNNs). An analog circuit has been proposed to implement the tanh by transforming the PCM <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">I-V</i> characteristics. The results indicate that the implemented tanh based on PCM presents a low root-mean-square error to the ideal tanh. Furthermore, the performance of the implemented tanh was verified in a classical deep neural network (DNN) with the handwritten digit recognition task. The implemented tanh with realistic characteristics presents a training accuracy (>90%) and good precision for repeatability inference in DNN. This work provides effective means for implementing activation functions in ANN based on analog memory.
27
Gradient-based learning applied to document recognition
Yann LeCun, Léon Bottou, Yoshua Bengio et al. · Proceedings of the IEEE · 1998 · 58.2K citations · Full text
Fully hardware-implemented memristor convolutional neural network
Peng Yao, Huaqiang Wu, Bin Gao et al. · Nature · 2020 · 2.1K citations
Engineering, Computer Engineering, Electrical Engineering +12
Equivalent-accuracy accelerated neural-network training using analogue memory
Stefano Ambrogio, Pritish Narayanan, Hsinyu Tsai et al. · Nature · 2018 · 1.1K citations · Full text
Stochastic phase-change neurons
Tomáš Tůma, Angeliki Pantazi, Manuel Le Gallo et al. · Nature Nanotechnology · 2016 · 1.1K citations