The effect of conversational agent skill on user behavior during deception

TLDR

Conversational agents are increasingly human‑like, yet their enhanced naturalness may provoke deceptive users to adopt strategic tactics that hinder deception detection. The study aims to determine how a CA’s conversational skill influences users’ deceptive behavior. The authors examined how varying CA conversational skill levels affect behavioral indicators of deception. Results show that deception cues vary with CA skill, and higher skill induces strategic behaviors that impair deception detection, suggesting that pursuing more human‑like interactions may be counter‑productive.

Abstract

Conversational agents (CAs) are an integral component of many personal and business interactions. Many recent advancements in CA technology have attempted to make these interactions more natural and human-like. However, it is currently unclear how human-like traits in a CA impact the way users respond to questions from the CA. In some applications where CAs may be used, detecting deception is important. Design elements that make CA interactions more human-like may induce undesired strategic behaviors from human deceivers to mask their deception. To better understand this interaction, this research investigates the effect of conversational skill—that is, the ability of the CA to mimic human conversation—from CAs on behavioral indicators of deception. Our results show that cues of deception vary depending on CA conversational skill, and that increased conversational skill leads to users engaging in strategic behaviors that are detrimental to deception detection. This finding suggests that for applications in which it is desirable to detect when individuals are lying, the pursuit of more human-like interactions may be counter-productive.