Large-Scale Monitoring for Cyber Attacks by Using Cluster Information on Darknet Traffic Features

Abstract

This paper presents a machine learning approach to large-scale monitoring for malicious activities on Internet. In the proposed system, network packets sent from a subnet to a darknet (i.e., a set of unused IPs) are collected, and they are transformed into 27-dimensional TAP (Traffic Analysis Profile) feature vectors. Then, a hierarchical clustering is performed to obtain clusters for typical malicious behaviors. In the monitoring phase, the malicious activities in a subnet are estimated from the closest TAP feature cluster. Then, such TAP feature clusters for all subnets are visualized on the proposed monitoring system in real time. In the experiment, we use a big data set of 303,733,994 darknet packs collected from February 1st to February 28th, 2014 (28 days) for monitoring. As a result, we can successfully detect an indication of the pandemic of a new malware, which attacked to the vulnerability of Synology NAS (port 5,000/TCP).