An Efficient Mutation-Based Fuzz Testing Approach for Detecting Flaws of Network Protocol

Abstract

Security flaws existed in protocol implementations might be exploited by malicious attackers and the consequences can be very serious. Therefore, detecting vulnerabilities of network protocol implementations is becoming a hot research topic recently. However, protocol security test is a very complex, challenging and error-prone task, as constructing test packets manually or randomly are not practical. This paper presents an efficient mutation-based approach for detecting implementation flaws of network protocol. Compared with other protocol testing tools, our approach divides the procedure of protocol testing into many phases, and flexible design can cover many testing cases for the protocol implementations under testing, and could apply for testing various protocol implementations quite easily. Besides, this approach is more comprehensible that makes the protocol security test easier to carry out. To assess the usefulness of this approach, several experiments are performed on four FTP server implementations and the results showed that our approach can find flaws of protocol implementation very easily. The method is of the important application value and can improve the security of network protocols.