IEEE Access · 2023 · 11 citations · 25 references
Social SciencesEducational PsychologyPsychologyComputer VisionMachine LearningStudent EngagementEye Movement AnalysisFace PerceptionFacial ExpressionFacial PerceptionLearner Engagement ModelingReaction TimeFacial Action UnitsFacial Expression AnalysisFacial Expression RecognitionGradient BoostingLight Gradient Boosting MachineStudent Attention ModelingTask-irrelevant DistractorsVisual Reaction Time
The present study aimed to develop a method for estimating students’ attentional state from facial expressions during online lectures. We estimated the level of attention while students watched a video lecture by measuring reaction time (RT) to detect a target sound that was irrelevant to the lecture. We assumed that RT to such a stimulus would be longer when participants were focusing on the lecture compared with when they were not. We sought to estimate how much learners focus on a lecture using RT measurement. In the experiment, the learner’s face was recorded by a video camera while watching a video lecture. Facial features were analyzed to predict RT to a task-irrelevant stimulus, which was assumed to be an index of the level of attention. We applied a machine learning method, light Gradient Boosting Machine (LightGBM), to estimate RTs from facial features extracted as action units (AUs) corresponding to facial muscle movements by an open-source software (OpenFace). The model obtained using LightGBM indicated that RTs to the irrelevant stimuli can be estimated from AUs, suggesting that facial expressions are useful for predicting attentional states while watching lectures. We re-analyzed the data while excluding RT data with sleepy faces of the students to test whether decreased general arousal caused by sleepiness was a significant factor in the RT lengthening observed in the experiment. The results were similar regardless of the inclusion of RTs with sleepy faces, indicating that facial expression can be used to predict learners’ level of attention to video lectures.
25
Michael I. Posner · Quarterly Journal of Experimental Psychology · 1980 · 9.6K citations
Medicine, Neuroscience, Psychology +14
LightGBM: A Highly Efficient Gradient Boosting Decision Tree
Guolin Ke, Qi Meng, Thomas Finley et al. · HAL (Le Centre pour la Communication Scientifique Directe) · 2017 · 9.5K citations · Full text
Electrical Signs of Selective Attention in the Human Brain
Steven A. Hillyard, Robert F. Hink, Vincent L. Schwent et al. · Science · 1973 · 2K citations
OpenFace 2.0: Facial Behavior Analysis Toolkit
Tadas Baltrušaitis, Amir Zadeh, Yao Chong Lim et al. · 2018 · 1.8K citations