A Multidisciplinary Survey and Framework for Design and Evaluation of Explainable AI Systems

TLDR

The growing prevalence of AI in everyday life has increased the need for interpretable and accountable systems, prompting interdisciplinary research into XAI that faces challenges from divergent objectives and fragmented knowledge. This article surveys XAI research across machine learning, visualization, and human‑computer interaction to propose a framework that unifies design goals and evaluation methods. The authors categorize XAI design goals by user group and map them to appropriate evaluation methods, then develop a step‑by‑step framework that pairs design guidelines with evaluation techniques. The resulting framework includes ready‑to‑use tables of evaluation methods and recommendations tailored to specific XAI goals.

Abstract

The need for interpretable and accountable intelligent systems grows along with the prevalence of artificial intelligence ( AI ) applications used in everyday life. Explainable AI ( XAI ) systems are intended to self-explain the reasoning behind system decisions and predictions. Researchers from different disciplines work together to define, design, and evaluate explainable systems. However, scholars from different disciplines focus on different objectives and fairly independent topics of XAI research, which poses challenges for identifying appropriate design and evaluation methodology and consolidating knowledge across efforts. To this end, this article presents a survey and framework intended to share knowledge and experiences of XAI design and evaluation methods across multiple disciplines. Aiming to support diverse design goals and evaluation methods in XAI research, after a thorough review of XAI related papers in the fields of machine learning, visualization, and human-computer interaction, we present a categorization of XAI design goals and evaluation methods. Our categorization presents the mapping between design goals for different XAI user groups and their evaluation methods. From our findings, we develop a framework with step-by-step design guidelines paired with evaluation methods to close the iterative design and evaluation cycles in multidisciplinary XAI teams. Further, we provide summarized ready-to-use tables of evaluation methods and recommendations for different goals in XAI research.