A three-dimensional model of student interest during learning using multimodal fusion with natural sensing technology

Abstract

A student’s interest level can strongly affect the learning process, and thus, can be considered an important factor in the effort to improve learning. Presently, student interest is primarily assessed by administering questionnaires or conducting case analyses. However, this method cannot provide timely feedback in the learning environment to allow an instructor to make immediate improvements for a more effective learning process. Hence, we designed an intelligent analysis method to analyse student interest using multimodal natural sensing technology. In this study, we present a three-dimensional (3D) learning interest model designed from an educational psychology perspective which comprehensively describes student interest in a learning environment: cognitive attention, learning emotion and thinking activity. Multimodal data are compiled by head pose estimation, facial expression recognition and interactive data collection and interpreted based on this model. Then, multimodal data fusion is conducted to comprehensively gauge student interest. Experimental testing revealed that the proposed 3D model of learning interest could objectively reflect student interest, providing an effective basis for improving teaching in real time.