2012 · 16 citations · 13 references
EngineeringComputer ScienceRobotics EngineeringArtificial IntelligenceHuman-robot InteractionRoboticsConversational InterfacesConversational User InterfacesDialogue SystemsHuman-machine InterfacesHuman-robot Interaction DesignHuman-robot Interaction EvaluationConversational AgentsDialog SystemsEvaluation Of Dialogue SystemsMetricsSpoken Dialogue SystemsUser SatisfactionUser Satisfaction EvaluationUser Satisfaction Measurement
We present the design process, realization and evaluation of a robot system for nteractive object learning. The system-oriented evaluation, in particular, addresses an open problem for the evaluation of systems, where overall user satisfaction depends not only on the performance of the parts, but also on their combination, and on user behavior. Based on the PARADISE method known from spoken dialog systems, we have defined and applied internal and external metrics for fine-grained and largely automatable identification of such relationships. Through evaluation with n=28 subjects, indicator functions explaining up to 55% of variation in several satisfaction metrics were found. Furthermore, we demonstrate that the system's interaction style reduces the need for instruction and successfully recovers partial failures.
13
J. A. Nelder, R. W. M. Wedderburn · Journal of the Royal Statistical Society Series A (General) · 1972 · 7.2K citations
Marilyn Walker, Diane Litman, Candace Kamm et al. · 1997 · 132 citations · Full text