Resilient Chatbots

TLDR

Text-based conversational systems, or chatbots, have become popular, yet current natural language understanding technologies struggle with conversational complexities, leading to frequent breakdowns and negative user experiences. The study aims to investigate user preferences for eight repair strategies in chatbots, guided by communication theories. We conducted a scenario-based study with 203 Mechanical Turk workers to compare these repair strategies. Participants favored providing options and explanations, which signal initiative and offer actionable recovery, and our analysis revealed nuanced strengths and weaknesses of each strategy.

Abstract

Text-based conversational systems, also referred to as chatbots, have grown widely popular. Current natural language understanding technologies are not yet ready to tackle the complexities in conversational interactions. Breakdowns are common, leading to negative user experiences. Guided by communication theories, we explore user preferences for eight repair strategies, including ones that are common in commercially-deployed chatbots (e.g., confirmation, providing options), as well as novel strategies that explain characteristics of the underlying machine learning algorithms. We conducted a scenario-based study to compare repair strategies with Mechanical Turk workers (N=203). We found that providing options and explanations were generally favored, as they manifest initiative from the chatbot and are actionable to recover from breakdowns. Through detailed analysis of participants' responses, we provide a nuanced understanding on the strengths and weaknesses of each repair strategy.