An Empirical Evaluation of the Effectiveness of Attack Graphs and Fault Trees in Cyber-Attack Perception

Abstract

Perceiving and understanding cyber-attacks can be a difficult task. This problem is widely recognized and welldocumented, and more effective techniques are needed to aid cyber-attack perception. Attack modeling techniques (AMTs), such as attack graphs and fault trees, are useful visual aids that can aid cyber-attack perception; however, there is little empirical or comparative research which evaluates the effectiveness of these methods. This paper reports the results of an empirical evaluation between an adapted attack graph method and the fault tree standard to determine which of the two methods is more effective in aiding cyber-attack perception. An empirical evaluation (n = 63) was conducted through a 3 × 2 × 2 factorial design. Participants from computer-science and non-computerscience backgrounds were divided into an adapted attack graph and fault tree group and then asked to complete three tests which tested the ability to recall, comprehend, and apply the AMT. A mean assessment score (mas) was calculated for each test. The results show that the adapted attack graph method is more effective at aiding cyber-attack perception when compared with the fault tree method (p <; 0.01). Participants that have a computer science background outperformed other participants when using both methods (p <; 0.05). These results indicate that the adapted attack graph method can be an effective tool for aiding cyber-attack perception amongst experts. This paper underlines the need for further comparisons in a broader range of settings involving additional techniques, and makes several suggestions for further work.