Algorithm for Searching and Analyzing Abnormal Observations of Statistical Information Based on The Arnold – Kolmogorov – Hecht-Nielsen Theorem

Abstract

An algorithm for detecting runouts in statistical information is proposed in this article. The idea of the algorithm is based on the property of some neural networks to demonstrate a large error in examples during training, which are runouts. For example, if a perceptron-type neural network has a relatively small number of hidden neurons, and if there are relatively few runouts in the training sample, then the neural network usually demonstrates a higher training error after the training procedure on the examples that are runouts than on nonrunout examples. However, two extreme cases are possible. On the one hand, if a neural network has too many degrees of freedom, it is usually well trained and demonstrates small values of the training error in all examples during training, including examples that are runouts. This is why a neural network with a large number of hidden neurons is not suitable for detecting runouts. On the other hand, if a neural network has too few degrees of freedom, it will demonstrate large values of the error both in runout examples and in examples that are not runouts after the training procedure. As such, it is also not suitable for detecting runouts. According to the proposed algorithm, a special neural network is designed using the formula obtained based on the relation derived from the Arnold -Kolmogorov -Hecht-Nielsen theorem. This special neural network is designed only for detecting and identifying outliers. Another neural network is being designed or other analysis methods are used for further data analysis. The proposed algorithm is intended for nonlinear subject areas described by small volumes of statistical samples that do not necessarily satisfy the normal distribution law. The application of the algorithm turned out to be efficient in solving a wide range of problems from various subject areas, such as medicine, economics, forensics, etc.