The four dimensions of contestable AI diagnostics - A patient-centric approach to explainable AI

TLDR

The problem of explainability in AI decision‑making has attracted considerable attention in recent years. The authors propose that explainability in AI diagnostics should be framed as “effective contestability”, requiring patients to contest diagnoses through access to information on data use, biases, performance, and labor division, and they define thirteen specific informational requirements and examine their domain specificity. The study adopts a patient‑centric framework, arguing that contestability demands access to data‑use, bias, performance, and labor‑division information, and it operationalizes this by defining thirteen specific informational requirements. The authors find that contestability is a weaker requirement than some explainability criteria, introduces no poorly grounded double standards, and does not compromise AI system performance.

Abstract

The problem of the explainability of AI decision-making has attracted considerable attention in recent years. In considering AI diagnostics we suggest that explainability should be explicated as ‘effective contestability’. Taking a patient-centric approach we argue that patients should be able to contest the diagnoses of AI diagnostic systems, and that effective contestation of patient-relevant aspect of AI diagnoses requires the availability of different types of information about 1) the AI system's use of data, 2) the system's potential biases, 3) the system performance, and 4) the division of labour between the system and health care professionals. We justify and define thirteen specific informational requirements that follows from ‘contestability’. We further show not only that contestability is a weaker requirement than some of the proposed criteria of explainability, but also that it does not introduce poorly grounded double standards for AI and health care professionals’ diagnostics, and does not come at the cost of AI system performance. Finally, we briefly discuss whether the contestability requirements introduced here are domain-specific.