Automation-induced complacency for monitoring highly reliable systems: the role of task complexity, system experience, and operator trust

TLDR

The growing quantity and complexity of advanced automated systems has raised concerns about automation‑induced complacency, where operators struggle to monitor system status. This study examined how system reliability, monitoring complexity, operator trust, and system experience affect automation‑induced complacency. Participants performed a manually controlled flight task while monitoring simulated aircraft displays for failures across three sessions to assess detection of single automation failures. Higher system reliability, greater monitoring task complexity, and increased operator experience all reduced monitoring performance, while trust rose; thus, highly reliable systems can foster complacency and impair detection of unexpected states.

Abstract

The increase in quantity and complexity of advanced automated systems has generated new concerns surrounding automation-induced complacency, or the difficulties operators have monitoring the status of automated systems. The present investigation consists of two studies that assessed the impact of system reliability, monitoring complexity, operator trust, and system experience on automation-induced complacency. In both studies, participants operated a manually controlled flight task while monitoring several simulated aircraft displays for failures. The ability of operators to detect a single automation failure over three experimental sessions was also assessed. Results indicated that realistic levels of system reliability severely impaired an operator's ability to monitor effectively. Further, as system experience increased, operator monitoring performance declined. The results also indicated that the complexity of the monitoring task heavily influenced operator monitoring, with poorer performance associated with more cognitively demanding tasks. Finally, results from both studies indicated that operator trust increased and monitoring performance decreased as a function of increasing system reliability. These results suggest that for highly reliable systems, increasing task complexity and extensive experience may severely impair an operator's ability to monitor for unanticipated system states.