DARPA's Explainable Artificial Intelligence Program

TLDR

Machine learning has spurred widespread AI applications across many domains, yet these systems lack explainability for human users. The XAI program aims to develop AI systems whose models and decisions are understandable and trustworthy, requiring new learning methods, explanation interfaces, and psychological insights. XAI developer teams are creating explainable machine‑learning techniques, designing explanation interfaces, and applying psychological theories to build evaluation frameworks, while continuously testing how their explanation systems enhance user understanding, trust, and performance. By May 2018, the XAI teams had finished the first phase of the four‑year program.

Abstract

Dramatic success in machine learning has led to a new wave of AI applications (for example, transportation, security, medicine, finance, defense) that offer tremendous benefits but cannot explain their decisions and actions to human users. DARPA's explainable artificial intelligence (XAI) program endeavors to create AI systems whose learned models and decisions can be understood and appropriately trusted by end users. Realizing this goal requires methods for learning more explainable models, designing effective explanation interfaces, and understanding the psychologic requirements for effective explanations. The XAI developer teams are addressing the first two challenges by creating ML techniques and developing principles, strategies, and human‐computer interaction techniques for generating effective explanations. Another XAI team is addressing the third challenge by summarizing, extending, and applying psychologic theories of explanation to help the XAI evaluator define a suitable evaluation framework, which the developer teams will use to test their systems. The XAI teams completed the first of this 4‐year program in May 2018. In a series of ongoing evaluations, the developer teams are assessing how well their XAM systems' explanations improve user understanding, user trust, and user task performance.