Development and application of an algorithm for extracting multiple linear regression equations from artificial neural networks for nonlinear regression problems

Abstract

This paper discusses the development and application of a decomposition neural network rule extraction algorithm for nonlinear regression problems, the algorithm is called the piece-wise linear artificial neural network or PWL-ANN algorithm. Rules in the form of linear equations are generated by approximating the sigmoid activation functions of the hidden neurons in an artificial neural network (ANN). The developed algorithm was applied to nineteen datasets. The preliminary results showed that the algorithm gives satisfactory results on sixteen of the nineteen tested datasets and the results demonstrate high fidelity to the originally trained neural network models. By analyzing the values of R <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> given by the PWL approximation on the hidden neurons and the overall output, it is evident that there are more factors affecting the fidelity of the algorithm apart from the precision of the approximation of each individual node of the given ANN model. Nevertheless, the algorithm shows promising potential for application in engineering problems.