Effects of Explainable Artificial Intelligence on trust and human behavior in a high-risk decision task

TLDR

AI recommendations are critical in high‑risk decisions such as distinguishing edible from poisonous mushrooms. The study examined how explainable AI methods and an educational intervention influence trust and decision behavior in an AI‑assisted mushroom‑picking task. A 2×2 online experiment with 410 participants used a virtual mushroom‑picking task, where an AI app displayed image classifications and, in one condition, added attribution‑based and example‑based explanations, while another condition provided educational information about the AI. Participants receiving explanations performed better and exhibited more calibrated trust, whereas the educational intervention, domain knowledge, and AI knowledge had no effect on performance.

Abstract

Understanding the recommendations of an artificial intelligence (AI) based assistant for decision-making is especially important in high-risk tasks, such as deciding whether a mushroom is edible or poisonous. To foster user understanding and appropriate trust in such systems, we assessed the effects of explainable artificial intelligence (XAI) methods and an educational intervention on AI-assisted decision-making behavior in a 2 × 2 between subjects online experiment with N=410 participants. We developed a novel use case in which users go on a virtual mushroom hunt and are tasked with picking edible and leaving poisonous mushrooms. Users were provided with an AI-based app that showed classification results of mushroom images. To manipulate explainability, one subgroup additionally received attribution-based and example-based explanations of the AI's predictions; for the educational intervention one subgroup received additional information on how the AI worked. We found that the group that received explanations outperformed that which did not and showed better calibrated trust levels. Contrary to our expectations, we found that the educational intervention, domain-specific (i.e., mushroom) knowledge, and AI knowledge had no effect on performance. We discuss practical implications and introduce the mushroom-picking task as a promising use case for XAI research.