Automated measurement of children's facial expressions during problem solving tasks

Abstract

There has been growing recognition of the importance of adaptive tutoring systems that respond to the student's emotional and cognitive state. However little is known about children's facial expressions during a problem solving task. What are the actual signals of boredom, interest, or confusion in real, spontaneous behavior of students? The field also is in need of spontaneous datasets to drive automated recognition of these states. This project aims to collect, measure, and describe spontaneous facial expressions of children during problem solving. We apply the computer expression recognition toolbox (CERT) to videos of behavioral data from children between the ages of 3 and 9, particularly focusing on the spontaneous expressions of uncertainty. From the Facial Action outputs, we analyze changes in facial expression during problem solving, and differences in expression between correct and incorrect trials. Moreover, we demonstrate differences in expression dynamics between older and younger children during problem solving. Future work examines differences between facial action and other modalities such as voice pitch.