Bootstrapping a data mining intrusion detection system

Abstract

The application of data mining techniques in intrusion detection has received a lot of attention lately. Most of the approaches require of a training phase based on the availability of labelled data, where the labels indicate whether the points correspond to normal events or attacks. Unfortunately, this labelled data is not readily available in practice. In this paper we present a novel method based in intersecting segments of unlabelled data and using the intersection as the base data for unsupervised learning (clustering). The clustering algorithm, along with a method to find outliers with respect to the base clusters form the basis for separation of unlabelled data into groups of points that are normal (attack-free) and points that correspond to attacks. We show that the technique is very sucessful in separating points of the data sets of the DARPA, Lincoln Labs evaluation of 1999.