"Like Having a Really Bad PA"

TLDR

Conversational agents have become ubiquitous in everyday life, with major tech companies embedding proprietary CAs, yet HCI research has focused on multimodality rather than dialogue, leaving a gap in understanding how these interfaces are used daily. The study investigates interactional factors affecting everyday use of conversational agents through interviews with 14 users. The authors apply Norman’s gulfs of execution and evaluation framework to interpret the interview findings and inform future system design. Users’ expectations are misaligned with system capabilities, especially regarding machine intelligence, functionality, and goals.

Abstract

The past four years have seen the rise of conversational agents (CAs) in everyday life. Apple, Microsoft, Amazon, Google and Facebook have all embedded proprietary CAs within their software and, increasingly, conversation is becoming a key mode of human-computer interaction. Whilst we have long been familiar with the notion of computers that speak, the investigative concern within HCI has been upon multimodality rather than dialogue alone, and there is no sense of how such interfaces are used in everyday life. This paper reports the findings of interviews with 14 users of CAs in an effort to understand the current interactional factors affecting everyday use. We find user expectations dramatically out of step with the operation of the systems, particularly in terms of known machine intelligence, system capability and goals. Using Norman's 'gulfs of execution and evaluation' [30] we consider the implications of these findings for the design of future systems.