SimSensei kiosk: a virtual human interviewer for healthcare decision support

Abstract

We present SimSensei Kiosk, an implemented virtual human interviewer designed to create an engaging face-to-face inter-action where the user feels comfortable talking and sharing information. SimSensei Kiosk is also designed to create in-teractional situations favorable to the automatic assessment of distress indicators, defined as verbal and nonverbal behav-iors correlated with depression, anxiety or post-traumatic stress disorder (PTSD). In this paper, we summarize the de-sign methodology, performed over the past two years, which is based on three main development cycles: (1) analysis of face-to-face human interactions to identify potential distress indicators, dialogue policies and virtual human gestures, (2) development and analysis of a Wizard-of-Oz prototype sys-tem where two human operators were deciding the spoken and gestural responses, and (3) development of a fully au-tomatic virtual interviewer able to engage users in 15-25 minute interactions. We show the potential of our fully auto-matic virtual human interviewer in a user study, and situate its performance in relation to the Wizard-of-Oz prototype.