Protocol reverse based on hierarchical clustering and probability alignment from network traces

Abstract

Protocol reverse analysis is a research hotspot in the field of information confrontation and network security. Inferring the format of unknown binary protocols from network traces is one challenge task in protocol reverse engineering. In this study, a protocol reverse method for binary protocol which combined with hierarchical clustering and probability alignment is proposed. First, the method classifies the frames with same format into one set based on hierarchical clustering. Second, probability alignment algorithm is adopted to align the corresponding fields in protocol frame accurately. Finally, according to the characteristics of the statistical data, the boundary of the adjacent fields in the protocol is determined, and the protocol format is extracted according to the field boundary. The experimental results show that the method can effectively identify the format specification of the binary protocol and the semantic specification of some fields.