Automating mimicry attacks using static binary analysis

TLDR

Intrusion detection systems that monitor system‑call sequences now incorporate program counter and stack configuration to better characterize behavior, yet the extent to which attackers can still evade such enhanced detection remains unclear. This work introduces a novel technique that transforms the challenge of bypassing these advanced detection features into a conventional mimicry attack. Using symbolic execution on Intel x86 binaries, the authors built a static analysis tool that identifies control‑flow‑redirecting instructions and computes the necessary environment modifications, enabling an attacker to execute each system call in the correct context by manipulating code pointers. The tool successfully exploited three vulnerable programs and evaded detection by state‑of‑the‑art system‑call monitors, and its applicability was verified on three real‑world applications.

Abstract

Intrusion detection systems that monitor sequences of system calls have recently become more sophisticated in defining legitimate application behavior. In particular, additional information, such as the value of the program counter and the configuration of the program's call stack at each system call, has been used to achieve better characterization of program behavior. While there is common agreement that this additional information complicates the task for the attacker, it is less clear to which extent an intruder is constrained. In this paper, we present a novel technique to evade the extended detection features of state-of-the-art intrusion detection systems and reduce the task of the intruder to a traditional mimicry attack. Given a legitimate sequence of system calls, our technique allows the attacker to execute each system call in the correct execution context by obtaining and relinquishing the control of the application's execution flow through manipulation of code pointers. We have developed a static analysis tool for Intel x86 binaries that uses symbolic execution to automatically identify instructions that can be used to redirect control flow and to compute the necessary modifications to the environment of the process. We used our tool to successfully exploit three vulnerable programs and evade detection by existing state-of-the-art system call monitors. In addition, we analyzed three real-world applications to verify the general applicability of our techniques.