The impact of chatbot conversational skill on engagement and perceived humanness

TLDR

Conversational agents, often called chatbots, use natural language processing to engage users in applications such as support and personal assistance, yet little research has examined how improving their conversational skill affects user perceptions. The study investigates how conversational skill influences perceived social presence and anthropomorphism of a chatbot using Social Presence Theory. The authors conducted studies with 450 participants interacting with chatbots of differing conversational skill levels. Participants perceived higher conversational skill as increasing social presence and anthropomorphism, advancing understanding of human–computer interaction beyond technical aspects.

Abstract

Conversational agents (CAs)—frequently operationalized as chatbots—are computer systems that leverage natural language processing to engage in conversations with human users. CAs are often operationalized as chatbots which are used for many applications including technical support, customer service, and digital personal assistants. Despite their widespread use, little research to date has investigated how improving the conversational skill of a CA impacts user perceptions of the agent. To elucidate this relationship, this research uses Social Presence Theory to describe how conversational skill influences perceived social presence and ultimately anthropomorphism of a chatbot. We conducted a series of studies in which 450 participants interacted with CAs exhibiting varying levels of conversational skill. We show that people perceive a more skilled CA to be more socially present and anthropomorphic than a less skilled CA. This research advances the knowledge of computer-human interface in information systems, as CA research to date has largely focused on the technical challenges rather than the behavioral questions of how users interact with CAs.