2024 · 21 citations · 25 references
ManagementPsychologySocial PsychologyArtificial IntelligencePsychometricsTeamworkTeam CoordinationTeam Decision-makingTeam EffectivenessTeam PerformanceCooperativityHuman-ai InteractionPilot Study DesignTask Performance AnalysisTask Performance MetricsTeam Performance AssessmentUser Study Evaluation
As we shift to designing AI agents as teammates rather than tools, the social aspects of human-AI interaction become more pronounced. Consequently, to develop agents that are able to navigate the social dynamics that accompany cooperative teamwork, evaluation criteria that refer only to objective task performance will not be sufficient. We propose perceived cooperativity and teaming perception as subjective metrics for investigating successful human-AI teaming. Corresponding questionnaire scales were developed and tested in a pilot study employing the collaborative card game Hanabi, which has been identified as a unique setting for investigating human-AI teaming. Preliminary descriptive results suggest that rule-based and reinforcement learning-based agents differ in terms of perceived cooperativity and teaming perception. Future work will extend the results in a large user study to psychometrically evaluate the scales and test a conceptual framework that includes further aspects related to social dynamics in human-AI teaming.
25
Human-level control through deep reinforcement learning
Volodymyr Mnih, Koray Kavukcuoglu, David Silver et al. · Nature · 2015 · 30K citations
Language Models are Few-Shot Learners
T. B. Brown, Benjamin Mann, Nick Ryder et al. · arXiv (Cornell University) · 2020 · 3K citations · Full text
Clifford Nass, Jonathan Steuer, Ellen R. Tauber · 1994 · 1.8K citations