Manufacturing cheap, resilient, and stealthy opaque constructs

TLDR

Software is often distributed in forms isomorphic to source code, such as Java bytecode, which are easy to decompile and thus expose programs to malicious reverse‑engineering attacks. This work designs a Java code obfuscator that applies transformations to produce an equivalent program that is harder to reverse engineer, particularly by obfuscating control flow. The obfuscator’s transformations are assessed on potency, resilience, cost, and stealth, and many rely on opaque predicates—boolean expressions known to the obfuscator but hard for automated deobfuscators to evaluate. We demonstrate how to build opaque predicates that are resilient, inexpensive, and stealthy by exploiting the intractability of static‑analysis problems such as alias analysis.

Abstract

It has become common to distribute software in forms that are isomorphic to the original source code. An important example is Java bytecode. Since such codes are easy to decompile, they increase the risk of malicious reverse engineering attacks.In this paper we describe the design of a Java code obfuscator, a tool which - through the application of code transformations - converts a Java program into an equivalent one that is more difficult to reverse engineer.We describe a number of transformations which obfuscate control-flow. Transformations are evaluated with respect to potency (To what degree is a human reader confused?), resilience (How well are automatic deobfuscation attacks resisted?), cost (How much time/space overhead is added?), and stealth (How well does obfuscated code blend in with the original code?).The resilience of many control-altering transformations rely on the resilience of opaque predicates. These are boolean valued expressions whose values are known to the obfuscator but difficult to determine for an automatic deobfuscator. We show how to construct resilient, cheap, and stealthy opaque predicates based on the intractability of certain static analysis problems such as alias analysis.