Human confidence in artificial intelligence and in themselves: The evolution and impact of confidence on adoption of AI advice

TLDR

Artificial intelligence promises to aid human decision‑making, yet inappropriate acceptance or rejection of AI suggestions can have severe consequences, highlighting a gap in understanding human trust in AI. This study investigates how confidence in AI and confidence in oneself evolve and influence human decisions regarding AI advice. Using a cognitive experiment and a quantitative model, the authors examine how positive and negative experiences shape these confidences and the resulting decision outcomes. Results show that self‑confidence—not confidence in AI—drives acceptance or rejection of AI advice, that people often blame themselves and become trapped in a cycle of relying on poorly performing AI, underscoring the need to calibrate self‑confidence for effective AI‑assisted decision‑making.

Abstract

Artificial intelligence (AI) has shown its promise in assisting human decision-making. However, humans' inappropriate decision to accept or reject suggestions from AI can lead to severe consequences in high-stakes AI-assisted decision-making scenarios. This problem persists due to insufficient understanding of human trust in AI. Therefore, this research studies how two types of human confidence that affect trust, their confidence in AI and confidence in themselves, evolve and affect humans' decisions. A cognitive study and a quantitative model together examine how changing positive and negative experiences affect these confidences and ultimate decisions. Results show that human self-confidence, not their confidence in AI, directs the decision to accept or reject AI suggestions. Furthermore, this work finds that humans often misattribute blame to themselves and enter a vicious cycle of relying on a poorly performing AI. Findings reveal the need and provide insights to effectively calibrate human self-confidence for successful AI-assisted decision-making.