Explanation and trust: what to tell the user in security and AI?

TLDR

AI and information security both require explanations to build user trust, as expert systems must justify decisions and the public must understand system security. The study examines how explanations influence trust in computing systems. The authors conduct a literature‑based concept analysis grounded in system theory and actor‑network theory, applying the framework to expert systems and electronic voting. The analysis highlights ethical implications, including informed consent, dissent, and responsibility allocation.

Abstract

There is a common problem in artificial intelligence (AI) and information security. In AI, an expert system needs to be able to justify and explain a decision to the user. In information security, experts need to be able to explain to the public why a system is secure. In both cases, an important goal of explanation is to acquire or maintain the users' trust. In this paper, I investigate the relation between explanation and trust in the context of computing science. This analysis draws on literature study and concept analysis, using elements from system theory as well as actor-network theory. I apply the conceptual framework to both AI and information security, and show the benefit of the framework for both fields by means of examples. The main focus is on expert systems (AI) and electronic voting systems (security). Finally, I discuss consequences of the analysis for ethics in terms of (un)informed consent and dissent, and the associated division of responsibilities.