Investigating how speech and animation realism influence the perceived personality of virtual characters and agents

TLDR

Virtual character personality shapes how users perceive and engage with digital applications. The study aims to determine how speech and animation features influence perceived personality to guide intentional character design. Using performance‑capture data from unscripted actor conversations, the authors compare fully performance‑driven characters with those using generated gestures and synthesized speech, analyzing how each modality affects Big Five personality traits. Results show motion drives extraversion while speech drives agreeableness and emotional stability, indicating mixed effects of speech and motion on personality traits and providing guidance for XR developers.

Abstract

The portrayed personality of virtual characters and agents is understood to influence how we perceive and engage with digital applications. Understanding how the features of speech and animation drive portrayed personality allows us to intentionally design characters to be more personalized and engaging. In this study, we use performance capture data of unscripted conversations from a variety of actors to explore the perceptual outcomes associated with the modalities of speech and motion. Specifically, we contrast full performance-driven characters to those portrayed by generated gestures and synthesized speech, analysing how the features of each influence portrayed personality according to the Big Five personality traits. We find that processing speech and motion can have mixed effects on such traits, with our results highlighting motion as the dominant modality for portraying extraversion and speech as dominant for communicating agreeableness and emotional stability. Our results can support the Extended Reality (XR) community in development of virtual characters, social agents and 3D User Interface (3DUI) agents portraying a range of targeted personalities.