Outlier Detection for Monitoring Data Using Stacked Autoencoder

Abstract

Monitoring data contain the important status information of the monitored object, and are the basis for following data mining and analysis. However, the monitoring data usually suffer the pollution of the outliers, leading to negative effect on the subsequent data processing. To address the problem, this paper proposed an outlier detection method based on stacked autoencoder (SAE). SAE has a powerful capability of feature extraction and greatly preserves the original information of the data. The trained SAE by normal data can learn the characteristics of normal data. When a set of data with outliers are inputted to the trained network, there are larger reconstruction errors at the outliers between the original input data and the reconstructed data obtained by using the encoding parameters and the decoding parameter mapping, which provides a basis for locating outliers. Meanwhile, this paper introduced the Grubbs criterion and the PauTa criterion to identify the reconstruction errors corresponding to the outliers based on the traditional threshold method. The method can quickly isolate the abnormal data from the normal data according to the reconstruction error and the identification criterion. The effectiveness and superiority of the proposed method have been validated by experiment on real data and comparisons with traditional outlier detection algorithms.