Sensors · 2019 · 53 citations · 65 references
ComputingComputer ScienceRobotics EngineeringComputer VisionHuman-robot InteractionRoboticsAffective ComputingEmotion RecognitionFace RecognitionFacial ExpressionFeature DetectionObject DetectionAffective State InferenceAutomatic Landmark DetectionChild-robot InteractionEmotion Recognition In InteractionFacial Landmark DetectionLandmark DetectionLinear Discriminant AnalysisThermal Infrared Imaging
Child-Robot Interaction (CRI) has become increasingly addressed in research and applications. This work proposes a system for emotion recognition in children, recording facial images by both visual (RGB-red, green and blue) and Infrared Thermal Imaging (IRTI) cameras. For this purpose, the Viola-Jones algorithm is used on color images to detect facial regions of interest (ROIs), which are transferred to the thermal camera plane by multiplying a homography matrix obtained through the calibration process of the camera system. As a novelty, we propose to compute the error probability for each ROI located over thermal images, using a reference frame manually marked by a trained expert, in order to choose that ROI better placed according to the expert criteria. Then, this selected ROI is used to relocate the other ROIs, increasing the concordance with respect to the reference manual annotations. Afterwards, other methods for feature extraction, dimensionality reduction through Principal Component Analysis (PCA) and pattern classification by Linear Discriminant Analysis (LDA) are applied to infer emotions. The results show that our approach for ROI locations may track facial landmarks with significant low errors with respect to the traditional Viola-Jones algorithm. These ROIs have shown to be relevant for recognition of five emotions, specifically disgust, fear, happiness, sadness, and surprise, with our recognition system based on PCA and LDA achieving mean accuracy (ACC) and Kappa values of 85.75% and 81.84%, respectively. As a second stage, the proposed recognition system was trained with a dataset of thermal images, collected on 28 typically developing children, in order to infer one of five basic emotions (disgust, fear, happiness, sadness, and surprise) during a child-robot interaction. The results show that our system can be integrated to a social robot to infer child emotions during a child-robot interaction.
65
Robust Real-Time Face Detection
Paul Viola, Michael Jones · International Journal of Computer Vision · 2004 · 14.2K citations
Aleix M. Martı́nez, A.C. Kak · IEEE Transactions on Pattern Analysis and Machine Intelligence · 2001 · 3.2K citations
Gary Bradski · Medical Entomology and Zoology · 2000 · 3.1K citations
One millisecond face alignment with an ensemble of regression trees
Vahid Kazemi, Josephine Sullivan · 2014 · 2.8K citations · Full text
Emotion recognition in human-computer interaction
Roddy Cowie, Ellen Douglas‐Cowie, Nicolas Tsapatsoulis et al. · IEEE Signal Processing Magazine · 2001 · 2.6K citations · Full text
Computing, Linguistics, Psychology +17