Deep Autoencoding Gaussian Mixture Model for Unsupervised Anomaly Detection

TLDR

Unsupervised anomaly detection on high‑dimensional data relies on density estimation, yet prior methods that separate dimensionality reduction from density modeling suffer from inconsistent optimization and loss of essential information. The authors introduce the Deep Autoencoding Gaussian Mixture Model (DAGMM) to address this problem. DAGMM couples a deep autoencoder, which supplies a low‑dimensional representation and reconstruction error, with a Gaussian mixture model that is jointly optimized end‑to‑end using an auxiliary estimation network, eliminating the need for separate pre‑training. Experiments on public benchmarks show that DAGMM surpasses state‑of‑the‑art techniques, achieving up to a 14 % improvement in F1 score.

Abstract

Unsupervised anomaly detection on multi- or high-dimensional data is of great importance in both fundamental machine learning research and industrial applications, for which density estimation lies at the core. Although previous approaches based on dimensionality reduction followed by density estimation have made fruitful progress, they mainly suffer from decoupled model learning with inconsistent optimization goals and incapability of preserving essential information in the low-dimensional space. In this paper, we present a Deep Autoencoding Gaussian Mixture Model (DAGMM) for unsupervised anomaly detection. Our model utilizes a deep autoencoder to generate a low-dimensional representation and reconstruction error for each input data point, which is further fed into a Gaussian Mixture Model (GMM). Instead of using decoupled two-stage training and the standard Expectation-Maximization (EM) algorithm, DAGMM jointly optimizes the parameters of the deep autoencoder and the mixture model simultaneously in an end-to-end fashion, leveraging a separate estimation network to facilitate the parameter learning of the mixture model. The joint optimization, which well balances autoencoding reconstruction, density estimation of latent representation, and regularization, helps the autoencoder escape from less attractive local optima and further reduce reconstruction errors, avoiding the need of pre-training. Experimental results on several public benchmark datasets show that, DAGMM significantly outperforms state-of-the-art anomaly detection techniques, and achieves up to 14% improvement based on the standard F1 score.