Algorithms · 2022 · 10 citations · 31 references
ComputingComputer ScienceData ScienceArtificial IntelligenceData MiningMachine LearningAnomaly DetectionDimension ReductionDimensionality ReductionOutlier DetectionPattern RecognitionRepresentation LearningAutoencoder-based Anomaly DetectionAutoencodersHigh-dimensional Anomaly DetectionHigh-dimensional Outlier DetectionOutlier ScoringSubspace Anomaly DetectionSubspace IdentificationSubspace Outlier Detection
When dealing with high-dimensional data, such as in biometric, e-commerce, or industrial applications, it is extremely hard to capture the abnormalities in full space due to the curse of dimensionality. Furthermore, it is becoming increasingly complicated but essential to provide interpretations for outlier detection results in high-dimensional space as a consequence of the large number of features. To alleviate these issues, we propose a new model based on a Variational AutoEncoder and Genetic Algorithm (VAEGA) for detecting outliers in subspaces of high-dimensional data. The proposed model employs a neural network to create a probabilistic dimensionality reduction variational autoencoder (VAE) that applies its low-dimensional hidden space to characterize the high-dimensional inputs. Then, the hidden vector is sampled randomly from the hidden space to reconstruct the data so that it closely matches the input data. The reconstruction error is then computed to determine an outlier score, and samples exceeding the threshold are tentatively identified as outliers. In the second step, a genetic algorithm (GA) is used as a basis for examining and analyzing the abnormal subspace of the outlier set obtained by the VAE layer. After encoding the outlier dataset’s subspaces, the degree of anomaly for the detected subspaces is calculated using the redefined fitness function. Finally, the abnormal subspace is calculated for the detected point by selecting the subspace with the highest degree of anomaly. The clustering of abnormal subspaces helps filter outliers that are mislabeled (false positives), and the VAE layer adjusts the network weights based on the false positives. When compared to other methods using five public datasets, the VAEGA outlier detection model results are highly interpretable and outperform or have competitive performance compared to current contemporary methods.
31
Generative adversarial networks
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza et al. · Communications of the ACM · 2020 · 13.6K citations · Full text
Support Vector Data Description
David M. J. Tax, Robert P. W. Duin · Machine Learning · 2003 · 3.5K citations · Full text
Efficient algorithms for mining outliers from large data sets
Sridhar Ramaswamy, Rajeev Rastogi, Kyuseok Shim · 2000 · 1.1K citations · Full text