From ‘automation’ to ‘autonomy’: the importance of trust repair in human–machine interaction

TLDR

Modern human–technology interactions are shifting from tool use to relationships with autonomous entities, and experts warn that designing highly autonomous systems must prioritize appropriate human‑centered considerations. The article proposes a human‑centered approach that embeds trust repair into autonomous systems to keep future human–autonomy interactions centered on users’ needs and preferences. By adapting literature from industrial psychology, the authors propose a framework that infuses a unique human‑like ability—building and actively repairing trust—into autonomous systems. They present a model and research agenda to guide the design of autonomous systems with trust repair capabilities, urging practitioners to view technology as a human‑human relationship rather than a tool.

Abstract

Modern interactions with technology are increasingly moving away from simple human use of computers as tools to the establishment of human relationships with autonomous entities that carry out actions on our behalf. In a recent commentary, Peter Hancock issued a stark warning to the field of human factors that attention must be focused on the appropriate design of a new class of technology: highly autonomous systems. In this article, we heed the warning and propose a human-centred approach directly aimed at ensuring that future human-autonomy interactions remain focused on the user's needs and preferences. By adapting literature from industrial psychology, we propose a framework to infuse a unique human-like ability, building and actively repairing trust, into autonomous systems. We conclude by proposing a model to guide the design of future autonomy and a research agenda to explore current challenges in repairing trust between humans and autonomous systems. Practitioner Summary: This paper is a call to practitioners to re-cast our connection to technology as akin to a relationship between two humans rather than between a human and their tools. To that end, designing autonomy with trust repair abilities will ensure future technology maintains and repairs relationships with their human partners.