A digital circuit design of hyperbolic tangent sigmoid function for neural networks

Abstract

This paper presents a digital circuit design approach for a commonly used activation function, hyperbolic tangent sigmoid functions, for neural networks. Our design concept for such a nonlinear function is to approximate the function of its first-order derivative by piece-wise linear functions first, then to obtain the estimate of the original function by integrating the approximated function of the first-order derivative by a digital circuit. The average error and maximum error of the proposed approximation approach are in the order of 10 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">−3</sup> and 10 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">−2</sup> , respectively in the software simulation. The hardware implementation of the proposed method consumes only one multiplication and one addition/subtraction ALU with the aid of resource sharing. The performance of our circuit has been validated by a neural network for a system identification problem in the software simulation.