Unsupervised incremental sequence learning for insider threat detection

Abstract

Insider threat detection requires the identification of rare anomalies in contexts where evolving behaviors tend to mask such anomalies. This paper proposes and tests an incremental learning algorithm based on unsupervised learning that addresses this challenge by maintaining repetitive sequences in a compressed dictionary to identify anomaly over dynamic data streams of unbounded length. For unsupervised learning, compression-based techniques are used to model normal behavior sequences. The result is a classifier that exhibits substantially increased classification accuracy for insider threat streams relative to traditional static learning approaches and effectiveness over supervised learning approaches.