False positive elimination in intrusion detection based on clustering

Abstract

In order to solve the problem of high false positive in network intrusion detection systems, we adopted clustering algorithms, the K-means algorithm and the Fuzzy C Mean (FCM) algorithm, to identify false alerts, to reduce invalid alerts and to purify alerts for a better analysis. In this paper, we first introduced typical clustering algorithms, including the partition clustering, the hierarchical clustering, the density and grid clustering, and the fuzzy clustering, and then analyzed their feasibilities in security data processing. Furthermore, we introduced an intrusion detection framework, and tested the validity and feasibility of false positive elimination in intrusion detection. The process steps of false positive elimination were clearly described, and additionally, two typical clustering algorithms, the K-means algorithm and the FCM algorithm, were implemented for false alerts identification and filtration. Also, we defined three evaluation indexes: the elimination rate, the false elimination rate and the miss elimination rate. Accordingly, we used DARPA 2000 LLDOS1.0 dataset for our experiments, and adopted Snort as our intrusion detection system. Eventually, the results showed that the method proposed by us has a satisfactory validity and feasibility in false positive elimination, and the clustering algorithms we adopted can achieve a high elimination rate.