Explainable artificial intelligence for education and training

TLDR

Artificial intelligence has rapidly expanded across domains, offering significant benefits, yet its opaque decision‑making raises concerns about control and transparency. The study reviews XAI capabilities, limitations, and desiderata, compares perspectives of AIED and AI/ML researchers, and offers guidelines for integrating XAI into educational research. The authors analyze XAI methods that reveal black‑box operations and compare AI and XAI approaches from AIED and AI/ML research perspectives. Both AIED and AI/ML researchers seek better XAI tools, yet they target different users and prioritize distinct features, leading to varied examples of potential benefits.

Abstract

Researchers and software users benefit from the rapid growth of artificial intelligence (AI) to an unprecedented extent in various domains where automated intelligent action is required. However, as they continue to engage with AI, they also begin to understand the limitations and risks associated with ceding control and decision-making to not always transparent artificial computer agents. Understanding of “what is happening in the black box” becomes feasible with explainable AI (XAI) methods designed to mitigate these risks and introduce trust into human-AI interactions. Our study reviews the essential capabilities, limitations, and desiderata of XAI tools developed over recent years and reviews the history of XAI and AI in education (AIED). We present different approaches to AI and XAI from the viewpoint of researchers focused on AIED in comparison with researchers focused on AI and machine learning (ML). We conclude that both groups of interest desire increased efforts to obtain improved XAI tools; however, these groups formulate different target user groups and expectations regarding XAI features and provide different examples of possible achievements. We summarize these viewpoints and provide guidelines for scientists looking to incorporate XAI into their own work.