Detecting Kernel-Level Rootkits Through Binary Analysis

TLDR

Rootkits are tools that conceal intruders from users and administrators; while early rootkits modified user‑space utilities, recent kernel‑module rootkits alter kernel data structures such as the system‑call table without changing user‑space binaries. This paper introduces a binary‑analysis technique that determines at load time whether a kernel module’s behavior resembles that of a rootkit. The technique employs an abstract model of module behavior that remains stable despite minor binary changes. The method offers additional protection against kernel‑module rootkits and resists attempts to hide malicious behavior.

Abstract

A rootkit is a collection of tools used by intruders to keep the legitimate users and administrators of a compromised machine unaware of their presence. Originally, root-kits mainly included modified versions of system auditing programs (e.g., ps or netstat on a Unix system). However, for operating systems that support loadable kernel modules (e.g., Linux and Solaris), a new type of rootkit has recently emerged. These rootkits are implemented as kernel modules, and they do not require modification of user-space binaries to conceal malicious activity. Instead, these rootkits operate within the kernel, modifying critical data structures such as the system call table or the list of currently-loaded kernel modules. This paper presents a technique that exploits binary analysis to ascertain, at load time, if a module's behavior resembles the behavior of a rootkit. Through this method, it is possible to provide additional protection against this type of malicious modification of the kernel. Our technique relies on an abstract model of module behavior that is not affected by small changes in the binary image of the module. Therefore, the technique is resistant to attempts to conceal the malicious nature of a kernel module.