"An Ideal Human"

Abstract

Driven by state-of-the-art AI technologies, human-AI collaboration has become an important area in computer-supported teamwork research. While human-AI collaboration has been investigated in various domains, more research is needed to explore human perceptions and expectations of AI teammates in human-AI teaming. To achieve an in-depth understanding of how people perceive AI teammates and what they expect from AI teammates in human-AI teaming, we conducted a survey with 213 participants and a follow-up interview with 20 participants. Considering the context-dependency of teamwork, we chose to study human-AI teaming in the context of multiplayer online games as a case study. This study shows that people have mixed feelings toward AI teammates but hold a positive attitude toward future collaboration with AI teammates in general. Our findings highlight people's expectations for AI teammates in a rapidly changing collaborative environment (e.g., instrumental skills for in-game tasks, shared understanding between humans and AI, communication capabilities, human-like behaviors and performance), as well as factors that impact people's willingness to team up with AI teammates (e.g., pre-existing attitudes toward AI, previous collaboration experience with humans). We contribute to CSCW by shedding light on how AI should be structured in human-AI teaming to support highly complex collaborative activities in CSCW environments.