VMFence

Abstract

Intrusion Prevention System (IPS) has been an effective tool to detect and prevent unwanted attempts, which are mainly through network and system vulnerabilities, at accessing and manipulating computer systems. Intrusion detection and prevention are two main functions of IPS. As attacks are becoming massive and complex, the traditional centralized IPSes are incapable of detecting all those attempts. The existing distributed IPSes, mainly based on mobile agent, have some serious problems, such as weak security of mobile agents, response latency, large code size. In this paper, we propose a customized intrusion prevention system, VMFence, in distributed virtual computing environment to simplify the complexity of the management. In VMFence, the states of detection processes vary with those of Virtual Machines (VMs), which are described by Deterministic Finite Automata (DFA). The detection processes, each of which detects one virtual machine, reside in a privileged virtual machine. The processes run synchronously and outside of VMs in order to achieve high performance and security. The experimental results also show VMFence has higher detection efficiency than traditional intrusion detection systems and little impact on the performance of the monitored VMs.