Why Johnny can't encrypt: a usability evaluation of PGP 5.0

TLDR

Most computer security failures stem from user errors, yet security interfaces remain clumsy and confusing, raising questions about whether standard UI design techniques can address these problems. The study argues that effective security demands a distinct usability standard, not achievable by conventional consumer‑software UI design techniques. The authors conducted a cognitive‑walkthrough analysis and laboratory user test of PGP 5.0 to assess whether novices could successfully sign and encrypt email, thereby testing the hypothesis. The study revealed UI flaws that contribute to security failures and showed that most participants could not sign or encrypt a message within 90 minutes, concluding that PGP 5.0’s usability is insufficient for effective security.

Abstract

User errors cause or contribute to most computer security failures, yet user interfaces for security still tend to be clumsy, confusing, or near-nonexistent. Is this simply due to a failure to apply standard user interface design techniques to security? We argue that, on the contrary, effective security requires a different usability standard, and that it will not be achieved through the user interface design techniques appropriate to other types of consumer software. To test this hypothesis, we performed a case study of a security program which does have a good user interface by general standards: PGP 5.0. Our case study used a cognitive walkthrough analysis together with a laboratory user test to evaluate whether PGP 5.0 can be successfully used by cryptography novices to achieve effective electronic mail security. The analysis found a number of user interface design flaws that may contribute to security failures, and the user test demonstrated that when our test participants were given 90 minutes in which to sign and encrypt a message using PGP 5.0, the majority of them were unable to do so successfully. We conclude that PGP 5.0 is not usable enough to provide effective security for most computer users, despite its attractive graphical user interface, supporting our hypothesis that user interface design for effective security remains an open problem. We close with a brief description of our continuing work on the development and application of user interface design principles and techniques for security.