IEEE Transactions on Affective Computing · 2022 · 318 citations · 50 references
EducationComputer ScienceEducational TechnologyComputer VisionMachine LearningStudent EngagementEmotion RecognitionEmotional ExpressionEngagement In Online LearningFace RecognitionFacial ExpressionOnline Student EngagementEmotionsFace IdentificationFacial Expression AnalysisFacial Expression ClassificationFacial Expression ProcessingFacial Expression RecognitionFacial Feature ProcessingReal-time Emotion Recognition
In this article, behaviour of students in the e-learning environment is analyzed. The novel pipeline is proposed based on video facial processing. At first, face detection, tracking and clustering techniques are applied to extract the sequences of faces of each student. Next, a single efficient neural network is used to extract emotional features in each frame. This network is pre-trained on face identification and fine-tuned for facial expression recognition on static images from AffectNet using a specially developed robust optimization technique. It is shown that the resulting facial features can be used for fast simultaneous prediction of students’ engagement levels (from disengaged to highly engaged), individual emotions (happy, sad, etc.,) and group-level affect (positive, neutral or negative). This model can be used for real-time video processing even on a mobile device of each student without the need for sending their facial video to the remote server or teacher's PC. In addition, the possibility to prepare a summary of a lesson is demonstrated by saving short clips of different emotions and engagement of all students. The experimental study on the datasets from EmotiW (Emotion Recognition in the Wild) challenges showed that the proposed network significantly outperforms existing single models.
50
Adam: A Method for Stochastic Optimization
Diederik P. Kingma, Jimmy Ba · UvA-DARE (University of Amsterdam) · 2014 · 84.8K citations · Full text
VGGFace2: A Dataset for Recognising Faces across Pose and Age
Qiong Cao, Li Shen, Weidi Xie et al. · 2018 · 2.9K citations
OpenFace 2.0: Facial Behavior Analysis Toolkit
Tadas Baltrušaitis, Amir Zadeh, Yao Chong Lim et al. · 2018 · 1.8K citations