Analysis of TCP flow data for traffic anomaly and scan detection

Abstract

Scanning tools are commonly used by intruders for identifying vulnerable hosts and applications in a network. So from security perspective, to identify the attack in its initial stage and to minimize the impact of attack, it is important to detect scanning activities in a network. We have mainly considered TCP flow because most of the Internet application uses it as a transport protocol. Traditionally, TCP scan traffic detection uses either flag values in the TCP packet header or statistical properties of the connection parameter like number of failed connection attempts. In this paper, we present a novel behaviour analysis of TCP traffic, where by using the flow characteristics, we identify anomalies and scan activities in a network or host. The proposed method provides a generic solution to SYN scan (half open), connect scan, FIN scan, Xmas scan and null scan. Results obtained from our method prove the detection capabilities and accuracy.