The Influence of Risky Conditions in Trust in Autonomous Systems

Abstract

In order to utilize the full range of benefits of autonomous systems, an understanding of how operators trust an automated system is vital. The level of risk in an environment is an important factor that many have suggested affects trust in automated and autonomous systems, but it has not been studied extensively. This study aims to explore the effect differing levels of risk can have on trust in an autonomous system with individuals that vary in their own risk profile. Using a UAV management task, participants worked with an autonomous teammate to protect an area from incoming enemies. Risk was assessed by how much money a participant stood to lose by not protecting the safe zone. Results partially support the hypothesis that trust decreases with increased risk, but results vary regarding behavioral trust and subjective trust. Overall, this experiment provides evidence that risk is an important situational factor that affects trust in automation.