Unsupervised Anomaly Detection with Variational Auto-Encoder and Local Outliers Factor for KPIs

Abstract

With the popularization of Internet application, online business systems is becoming more and more common. To ensure the normal running of online business systems, IT operations are required to closely monitor various KPIs (Key Performance Indicators), such as network throughput, page views, user’s order status, and discover the anomalies in the KPIs accurately in time. However, due to the diversity and complexity of anomalies, this task is confronted with great challenges, especially without labels. In this paper, we propose an unsupervised anomaly detection algorithm called LOF-VAE, based on LOF (Local Outliers Factor) and VAE (Variational Auto-Encoder). Our experiments using real-world data from large Internet companies show that, LOF-VAE’s best F-scores range from 0.89 to 0.99, and compared to the state-of-art unsupervised approach Donut, LOF-VAE improves the best F-score by 0.02 to 0.17. Even with low alert latency, LOF-VAE also performs well.