(Un)intended Consequences of AI Sales Assistants

Abstract

This study examines the design and implementation of an AI Sales Assistant (AISA) in several companies to understand the relationship between value sensitive design and successful implementation of AI assistants. While the AISA was designed to help and support collaboration, this study found that rather than supporting sales associates, the AISA evolved into a system to exclusively track and monitor them. This study reveals that while values important to the sales associates were considered in its design, a Value Sensitive Design approach to include all stakeholders and their values in the design process might have led to better outcomes.