An algorithm for anomaly-based botnet detection

TLDR

We present an anomaly‑based algorithm for detecting IRC‑based botnet meshes. The algorithm combines an IRC mesh detection component that identifies channels by IP channel names and aggregates host statistics—including a TCP work weight heuristic—and then ranks channels by scanner count to flag potential botnets. Deployment in PSU’s DMZ for over a year has reduced botnet client counts.

Abstract

We present an anomaly-based algorithm for detecting IRC-based botnet meshes. The algorithm combines an IRC mesh detection component with a TCP scan detection heuristic called the TCP work weight. The IRC component produces two tuples, one for determining the IRC mesh based on IP channel names, and a sub-tuple which collects statistics (including the TCP work weight) on individual IRC hosts in channels. We sort the channels by the number of scanners producing a sorted list of potential botnets. This algorithm has been deployed in PSU's DMZ for over a year and has proven effective in reducing the number of botnet clients.