Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance

TLDR

Many researchers motivate explainable AI with studies showing that human‑AI team performance on decision‑making tasks improves when the AI explains its recommendations, but prior work found such gains only when the AI alone outperforms both human and best team. The study asks whether explanations can produce complementary performance—team accuracy exceeding either solo human or AI—and whether explanatory methods can foster appropriate trust to achieve this. We performed mixed‑method user studies on three datasets where an AI with human‑level accuracy helped participants solve a task, with some conditions including AI self‑explanation. Complementary improvements from AI augmentation were observed, but explanations did not enhance them; instead, explanations merely increased the likelihood that humans would accept the AI’s recommendation regardless of its correctness.

Abstract

Many researchers motivate explainable AI with studies showing that human-AI team performance on decision-making tasks improves when the AI explains its recommendations. However, prior studies observed improvements from explanations only when the AI, alone, outperformed both the human and the best team. Can explanations help lead to complementary performance, where team accuracy is higher than either the human or the AI working solo? We conduct mixed-method user studies on three datasets, where an AI with accuracy comparable to humans helps participants solve a task (explaining itself in some conditions). While we observed complementary improvements from AI augmentation, they were not increased by explanations. Rather, explanations increased the chance that humans will accept the AI's recommendation, regardless of its correctness. Our result poses new challenges for human-centered AI: Can we develop explanatory approaches that encourage appropriate trust in AI, and therefore help generate (or improve) complementary performance?