A flow-based anomaly detection method using sketch and combinations of traffic features

Abstract

With the development of high-speed networks, the challenge of effectively analyzing the massive data source for anomaly detection and diagnosis is yet to be resolved. This paper proposes a new flow-based anomaly detection method based on summary data structures and combinations of traffic features. Using IPFIX flow records as input, parallel sketches are established for chosen traffic features respectively. For each sketch, we use Holt-Winters forecasting technique to achieve their forecast sketches and deviation matrixes. When the deviation exceeds a certain threshold, sub-alarms will be generated. According to the characteristics of various attacks and combinations of traffic features, sub-alarms can be merged into final alarms. While sketches of flows are being constructed, destination addresses are recorded in linked lists which are used to locate victims by a series of set operations. This method can not only detect the existence of anomalies in near real time, but can roughly indicate the anomaly types and locate abnormal addresses.