The explainability paradox: Challenges for xAI in digital pathology

TLDR

Digitised pathology workflows enable AI applications, but explainability is essential for safety and acceptance, and few user‑centric studies exist. We performed a mixed‑methods study of user interaction with state‑of‑the‑art AI explainability techniques in digital pathology. The study uncovers dilemmas for xAI developers and proposes empirically‑backed principles for safer, more effective design.

Abstract

The increasing prevalence of digitised workflows in diagnostic pathology opens the door to life-saving applications of artificial intelligence (AI). Explainability is identified as a critical component for the safety, approval and acceptance of AI systems for clinical use. Despite the cross-disciplinary challenge of building explainable AI (xAI), very few application- and user-centric studies in this domain have been carried out. We conducted the first mixed-methods study of user interaction with samples of state-of-the-art AI explainability techniques for digital pathology. This study reveals challenging dilemmas faced by developers of xAI solutions for medicine and proposes empirically-backed principles for their safer and more effective design.