In memory detection of Windows API call hooking technique

Abstract

API call hooking is a technique that malware researchers use to mine malware's API calls. These API calls is used to represent malware's behavior, for use in malware analysis, classification or detection of samples. In this paper, analysis of current Windows API call hooking techniques is presented where surprisingly, it was found that detection of each technique can be done trivially in memory. This could lead to malware being able to sense the presence of API call hooking techniques and modifying their behavior during runtime. Suggestions for a better API call hooking technique are presented towards the end of the paper.